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Abstract 


California’s students have generally been making slow gains on nationally representative 
tests over the better part of the last 10 years. While these improvements are important, the 
gaps between subgroups of students are large and overall achievement places California in the 
lowest 10 percent of states (NCES, 2015). Given that teachers matter more to student 
achievement than any other school-based factor (Rivkin, Hanushek, & Kain, 2000), 
one important policy lever for improving student learning is a strong teacher education 
system. If novice teachers are not prepared to meet all students’ needs on day one, students’ 
development is compromised. Novice teacher preparation is especially important to low- 
income, minority, and low-achieving students because their schools are disproportionately 
staffed by novices (Gagnon & Mattingly, 2012; Krei, 1998; Lankford, Loeb, & Wyckoff, 2002). 


To support California policymakers’ efforts to improve the preparation of novices, we 
describe California’s teacher education system, offering five criteria by which California might 
judge its success. We then evaluate the system’s alignment, what we know about the degree to 
which it produces beginners who are ready to teach on their first day, and the degree to which 
there is data that can support pipeline institutions’ efforts to improve the system. 


We find that the teacher pipeline should be thought of as a teacher education system 
comprised of institutions that provide novices with opportunities to learn the knowledge, skills, 
and capabilities in the California Standards for the Teaching Profession. K-12 schools play a 
large role in this system, teaching novices the K-12 curriculum, providing the learning context 
for all field placements, as well as providing mentor teachers for student teaching placements 
and induction mentors. 


Our evaluation of the teacher education system finds that policies and regulations are 
aligned with the state’s teaching standards at a high level. However, this high-level alignment 
masks wide variation in ground-level implementation. It is unclear whether this ground-level 
variation influences beginners’ preparedness. State reports and publicly available data suggest 
that no matter what path a novice takes through the teacher education system they are 
similarly prepared if they complete that pathway. They generally pass the required knowledge 
and practice-based assessments and they share the perception that they are well prepared to 
teach across the California standards. It is worth noting, however, that a sizable minority of 
multiple and single subject beginning teachers — 37 percent in 2016-17 — are teaching 
California’s students before completing a pathway, an assurance of adequate preparation. 


In contrast to these patterns, principal survey data collected for this analysis suggest 
there are differences in the perceived preparedness of novice teachers. On average, compared 
to novice teachers, principals report lower levels of teacher preparedness among novices and 
perceive there to be variation in novices’ preparedness across teaching practices. However, 
principals and teachers alike perceive teachers to have relatively lower levels of preparation to 
support students with special needs and relative strength in creating a positive classroom 
environment. 
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Current data systems provide a good deal of standardized information about the 
knowledge teachers have after completing a pathway, but less information and less systematic 
information about what teachers are able to do on their first day of teaching. In addition to 
sparse systematic information on novices’ capabilities, there are notable data constraints on 
the types of data available for improvement purposes. We conclude that policymakers and 
pipeline institutions do not yet have the information they need to rapidly improve the teacher 
education system. We consider how California might leverage its system of well-aligned 
standards, assessments, and requirements to create the additional data necessary to improve 
teacher preparedness across institutions in the teacher education system. 
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Introduction 


Between 2000 and 2011, California’s performance on the nation’s math and reading 
student assessment, the National Assessment of Educational Progress (NAEP), steadily 
improved. In more recent years, student achievement on the 4th grade math and 8th grade 
reading NAEP in California declined slightly; however, California continued to improve on 4th 
grade reading and 8th math NAEP scores between 2013 and 2017. NAEP is deliberately 
designed to be comparable over time, allowing states to compare their own policymaking 
efforts to a national student learning assessment. California’s improvements are important — 
between 2000 and 2017, the state’s average math and reading scores have increased even as 
the state has grown more racially and linguistically diverse, and weathered great economic 
upheaval that dramatically influenced public schools during the recession. Unfortunately, this 
progress is not the whole student achievement story. 


In 2017, only 29 percent of California’s 8th graders were proficient or advanced in 
mathematics on NAEP; 32 percent and 23 percent met the same standards in the most recent 
administrations of the reading and science NAEP, respectively (NCES, 2015, 2017). Performance 
of the state’s 4th graders was similar, with 31, 31, and 24 percent proficient or advanced in the 
most recent administrations of the mathematics, reading, and science NAEP, respectively. This 
level of performance places California among the six lowest-achieving jurisdictions in the 
country. Further, California’s gaps between economically advantaged and disadvantaged, black 
and white, and Latino and white students on the most recent administrations of the 
mathematics, reading and science NAEP remain unacceptably large, ranging between 22 and 38 
points (NCES, 2015, 2017). Compared to the other similarly populous states of Illinois, Texas, 
Florida, and New York, California regularly ranks lowest in terms of achievement (NCES, 2013).1 


Together, these facts suggest that while there has been movement in a positive 
direction, California’s student achievement is low and gaps are large. One way to accelerate 
improvement is for the state to effectively intervene in the most important in-school factor 
shaping student learning: teacher quality (Chetty, Friedman, & Rockoff, 2014; Clotfelter, Ladd, 
& Vigdor, 2007; Harris & Sass, 2007). Many efforts to improve teacher quality focus on 
professional development programs for in-service teachers, however, the evidence of those 
programs’ efficacy is mixed (Garet et al., 2016; Hill & Ball, 2004; Kraft, Blazar, & Hogan, in 
press). An exclusive focus on in-service teachers also misses a very important group of teachers, 
beginners.2 Nationally, beginning teachers disproportionally teach students of color and 
economically disadvantaged students (Gagnon & Mattingly, 2012) and they are significantly less 
effective than more experienced teachers (Darling-Hammond, 1999; Hanushek, Kain, O’Brien, & 
Rivkin, 2005; Nye, Konstantopoulos, & Hedges, 2004; Rockoff, 2004). Thus, improving initial 


1 Given the indelible impact of non-school factors on student achievement, including socioeconomic status, native 
language, and mobility, it is important to note that California also has the largest share of low-income students, 
English learners, and recent immigrant students compared to the other states in the aforementioned report. 

2 We use beginning teacher and novice teacher synonymously throughout this report. 
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teacher preparation is another important way policymakers can accelerate the improvement of 
teaching quality in California. 


The state’s initial teacher preparation system — a broad set of institutions that extend 
beyond teacher preparation programs (TPPs) — impacts thousands of teachers every year. In 
2016-17 alone, California issued 13,447 new general education preliminary teaching credentials 
to both in- and out-of-state prepared teachers.3 This paper describes the initial teacher 
preparation system — its substantive focus, alignment, and attendant policies and regulations. 
We evaluate the existing evidence regarding beginners’ preparedness and finally describe the 
nature and quality of data produced by the teacher education system. Based on these analyses 
we consider a handful of illustrative policy questions, investigating the degree to which they 
can be answered with currently available data. Based on these analyses, we suggest how 
California might build on system strengths and provide pipeline institutions and policymakers 
the strategic data they need to improve the teacher education system. 


The Teacher Pipeline as Teacher Education System 


If you ask a person where teachers learn to teach, they often respond with “their 
teacher preparation program.” This is a reasonable response. Certainly a teacher learns 
strategies such as organizing students into groups or designing appropriate lesson plans in her 
TPP. But many people assume that teacher preparation happens almost exclusively in TPPs. In 
fact, states regulate and ask many questions of these TPPs — are they accredited? What are the 
GPA requirements for admission? What are their students’ pass rates on certification tests? 
These are all important questions whose answers should be known. But the answers to these 
questions miss an important reality — teacher preparation involves many different institutions. 
The preparation of teachers does not take place inside a single program; it involves a system of 
educational institutions. We need only carry out a two-question thought experiment to 
understand this reality. 


First, where does a student teacher have the opportunity to work with K-12 students? 
Second, where does a math teacher learn the math she teaches? In the first case, a student 
teacher works with K-12 students outside of her TPP, in California’s K-12 classrooms. She likely 
does so early in her practice placements, perhaps when she tutors a small group of students, or 
later and more intensively, during her student teaching placement.4 On the second question, a 
math teacher learns mathematics in her own K-12 schooling experience and in her 
undergraduate math department, if she was a math major. If the teacher seeks elementary 


3 This includes multiple- and single-subject credentials only, including both traditional and intern credentials. This 
does not include instructional specialist credentials. 

4 All California-prepared teachers must “complete a teacher prep program including successful student teaching.” 
However, California has no requirement that out-of-state or internationally prepared teachers must complete a 
student teaching experience. To the degree this occurs, it is regulated by accreditation agencies and other 
country’s or state’s laws. Most states (41 of 50) have requirements regarding the length of student teaching, and 
15 states require additional clinical experiences (EPE Research Center, 2012). 
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certification, there will be a heavy reliance on her learning of topics such as fractions, numbers 
and operations, and measurement when she was a student in K-12 classrooms. A secondary 
math teaching candidate might learn more about geometry and calculus in her university math 
major than an elementary candidate, but her K-12 education will dramatically shape the quality 
and depth of her knowledge of K-12 math topics such as quadratic equations and algebra. 


These are relatively clear examples of two important learning opportunities all 
beginning teachers must successfully engage — learning to work with K-12 students and learning 
the subject matter. And they both occur in institutions outside of the TPP. This brief thought 
experiment demonstrates that teacher preparation is not confined to TPPs. Preservice teachers 
need to learn a wide range of technical, subject matter, and social skills in order for them to be 
effective beginning teachers, and that learning happens within and outside of TPPs as well as in 
collaborative arrangements between TPPs and other institutions. 


If we shift from thinking about teacher preparation as what happens inside of relatively 
brief programs to the fuller range of learning experiences that happen over time in multiple 
institutions across states and countries, it becomes clear that policymakers should think about 
teacher preparation as a system. That teacher education learning system can be conceptualized 
as a Set of learning opportunities provided by institutions, which are regulated by assessments 
as well as requirements. Those assessments and requirements, which come in the form of laws, 
statutes, rules, regulations, and policies, are designed to ensure every child has a well-prepared 
teacher. See Figure 1 for the institutions that participate in California’s teacher education 
system. 


Shifting from thinking about the policy target as the teacher education system versus 
the preparation program helps us think critically about alternative solutions to teaching quality 
problems. For example, take teaching shortages in science, technology, engineering, and 
mathematics (STEM) as one problem that impacts the supply of high-quality teachers in those 
subject areas. A narrow program view would focus on getting more prospective teachers to 
pass the subject matter tests in order to be certified. A policy intervention might decrease the 
required passing scores on these tests in order to increase the tests’ pass rates, thereby 
increasing the number of certified STEM teachers. This is one potential, albeit narrow, policy 
solution. But if there are a relatively small number of teachers taking the tests, the policy’s 
impact might only produce a few additional certified teachers each year, and those teachers 
may be underqualified. 


Continuing with the STEM teacher shortage problem, a system-level view of the 
problem draws policy attention to at least three additional locations in the system in which to 
intervene to produce more STEM teachers — high schools, community colleges and other 
undergraduate institutions, and teacher induction programs. Policies in each location could be 
effective policy levers to improve the supply of qualified STEM teachers. At the high school 
level, there are large numbers of students interested in STEM who might be enticed into 
teaching through university scholarships with required teaching commitments. At the college 
and university level, there could be policies that educate STEM faculty members about current 
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teaching shortages and teaching as a profession as a way to increase the number of faculty 
members recommending teaching as a profession to STEM undergraduates. Policies aimed at 
university students who are already STEM majors but have not selected into teaching might 
offer forgivable loans for candidates with a bachelor’s degree in a STEM field who are 
interested in teaching and are willing to commit to teach in schools with STEM teaching 
shortages. And finally, looking at induction programs, there could be additional salary, 
mentoring, professional learning opportunities, or reduced teaching loads for STEM teachers 
who stay in teaching for some amount of time. Such investments during induction’s early years 
could reduce turnover, and therefore demand. 


Viewing the teacher pipeline as a learning system pushes stakeholders to think flexibly 
and rigorously about policy alternatives for teaching policy problems. With a system view, 
policymakers have a framework through which they can lay out policy alternatives that target 
actors and institutions at different points along the pipeline. Policymakers can then quantify the 
potential number of individuals who might be eligible for each policy alternative, specify the 
proportion of teachers who might reasonably be expected to be certified or retained by the 
policy, and determine the costs of each policy alternative. This type of data-based, system- 
minded policymaking is the kind most likely to improve California’s teaching workforce over 
time. Such a view also embraces the improvement science orientation that has begun to show 
some promise in education (Baron, 2017). This approach calls for identifying process 
breakdowns within a system, and engaging individuals responsible for these processes to use 
disciplined inquiry to improve organizational functioning (Bryk, Gomez, Grunow, & LeMahieu, 
2015). 


Taking this system-level view, our goal is to describe and understand California’s teacher 
education learning system in order to identify how it might be improved. We investigate six 
research questions. 


1. What do beginning teachers need to know and be able to do on their first day ina 
California classroom? 

2. What are the main institutions, regulations, assessments, and policies shaping the 

knowledge, skills, and capabilities of California’s beginning teachers? 

How should we judge the quality of California’s teacher education system? 

How aligned is California’s system? 

How well does California’s system prepare teachers for their first day? 

What is the nature of the data in California’s system and to what degree can it be used 

for systemwide improvement efforts and policymaking? 


Yea eS w 


California’s teacher education learning system supports the development of teachers 
who teach a wide range of subjects, grade levels, and students — teachers of all subjects in the 
elementary grades, teachers of one particular subject area in high schools, teachers who focus 
on students with special needs, and teachers who work with English learners, just to name a 
few. There are more than 12 types of preliminary credentials issued each year in California 
(CTC, 2017d) ranging from single-subject and multiple-subject credentials that are usually 
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required to teach secondary and elementary grades, to teaching credentials for designated 
subjects such as career and technical education and vocational education, to short-term staff 
permits, waivers, and provisional internship permits. There is a great deal of complexity and 
detail within the various types of certificates, and differences in learning opportunities across 
those certificates deserve our careful attention (e.g., Santibafiez, 2018). However, in order to 
make the system’s detail more manageable and understand the details of the learning system 
the state has designed, we narrow our analyses to the preparation of elementary and 
secondary teachers earning either a multiple-subject or single-subject credential either through 
a California TPP or intern program. This focus corresponds to 74 percent of new teaching 
credentials issued in California during the 2016-17 school year (CTC, n.d.c).56 We analyze the 
teacher education system from where it begins in K-12 schools through to the “preliminary” 
and “clear” teaching credential. The preliminary credential is the non-renewable credential that 
allows candidates to become a teacher of record, is only good for five years, and has additional 
educational requirements associated with it. The clear credential acknowledges all education 
and program requirements have been met. 


Before we begin, it is important to note that in 2016-17, California permitted 7,861 
individuals to teach California’s students without actually completing the requirements in the 
state’s articulated system via a permit, waiver, and/or intern credential (see Table 1). Thus, this 
report cannot speak to the preparation experiences these teachers have, nor the degree to 
which they are prepared to support K-12 student development. Some research has shown that 
teachers who enter classrooms unprepared, teaching on credentials that do not meet all of the 
requirements of a teaching credential, are less effective than fully credentialed teachers (e.g., 
Goldhaber & Brewer, 2000; Darling-Hammond, Berry, & Thoreson, 2001), although the specifics 
of subject matter, credential, and modelling assumptions matter (Wayne & Youngs, 2003). 
Additional research might productively investigate the nature and quality of the learning 
opportunities that teachers on permits, waivers, and intern credentials have. 


The Required Knowledge, Skills, and Capabilities for Day One 


Understanding the teacher education system in California requires we first understand 
what beginners should know and be able to do once they have completed participation in that 
system. Beginning in the 1990s, the California Commission on Teacher Credentialing (CTC) laid 
out a consensus vision of the knowledge, skills, and capabilities (KSCs) all teachers should have 
in the California Standards for the Teaching Profession (hereafter referred to as the CSTPs) 
(CTC, 1997). In 2009, the CSTPs were revised and updated to reflect the increasingly diverse 
student population in California as well as a 21st century perspective that recognizes the role 
technology plays in society and the increasingly connected global community (CTC, 2009). The 
CSTPs apply to all phases in a teacher’s development, from the content she learns as an 


5 Of all multiple-subject or single-subject preliminary credentials issued in 2016-17, 17.4 percent were 
recommended by out-of-state or out-of-country preparation programs. 

6 The remaining 26 percent of preliminary credentials issued in California were education specialist certificates, 
which allow beginners to teach students with special needs. 
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undergraduate student to the pedagogical capabilities and specialized teaching knowledge she 
develops in her student teaching experiences and induction program all the way to the KSCs 
she learns through the professional learning that occurs across the remaining years of her 
teaching career.7 


A Professional Vision of Teaching 


To understand what beginning teachers are expected to know and be able to do, it is 
helpful to have a common reference for what teaching at any point along a teacher’s career 
might look like. Let us take a typical first five minutes in a hypothetical ninth grade mathematics 
classroom. 


Mrs. Jones writes on the smartboard at the front of the room as roughly 25 students of 
various racial, ethnic, and language backgrounds walk between desks. All of the students 
speak English well, but Mrs. Jones knows that Sara and Chris still struggle in her class, in 
part because as English language learners, their academic vocabulary in mathematics is 
not yet strong. Additional students enter the room, find their seats and open their 
backpacks, setting their homework in the blue basket on Mrs. Jones’ desk. The group 
swells to 32 students and as the last seven students are entering, turning in homework 
and taking out their books, the bell rings. Jose approaches Mrs. Jones and asks to speak 
to her privately. She squeezes his shoulder, looks up and sees Sara talking to her friend in 
the next desk, and notices one tear slowly sliding over Sara’s cheek. Mrs. Jones looks 
back at Jose and asks if they can speak after class. He nods and he takes his seat. 


The learning objective is written on the board “The student will be able to graph 
quadratic functions from quadratic equations that are in different forms.” Most students 
appear to be copying it in their notebooks. Mrs. Jones walks to stand beside Sara’s desk, 
looking at the board, and Mrs. Jones begins. 


Mrs. Jones: John (who is not raising his hand), what does this learning objective make 
you remember or wonder? 


John: It makes me think of x? + 2x = 0 and x?+2x-3 = 0 
Mrs. Jones: Why did you think of those two equations? 
John: They are different forms aren’t they? 


(Sara raises hand) 


Mrs. Jones: Sara has a response to that question | think. 

Sara: No, those are not different forms. They both equal zero so they are the 
same form. But they do have different terms. 

Mrs. Jones: Um. 


7 The CSTPs describe accomplished teaching practice; however, accomplished practice draws on a wide variety of 
knowledge, skills, and capabilities developed across the teacher learning pipeline. 
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(Chris raises his hand. Mrs. Jones nods to him.) 


Chris: I’m gonna add another different thought. That [learning] objective reminds 
me that I’m not sure of the difference between a function and an equation. 
Jose: (who does not raise his hand) Yeah, we should go over that, but | think we 


should say what a term is. Because Sara said term and form. 


(Mrs. Jones has walked to the smartboard and written the words function, equation, 
term, form) 


Mrs. Jones: Alright. Thank you everyone. Who can help us with some definitions and 
examples? Let’s start with a function. (Jose raises his hand.) Jose? 


As is clear from this brief example, classrooms have students with different needs and 
ideas. Mrs. Jones must make moment-to-moment decisions in this complex teaching and 
learning environment. For example, in the moment Jose asked to speak to her privately, Mrs. 
Jones had to decide whether to take a few moments to speak to him, ask Sara if she is alright, 
or begin class right away. Her decision was likely based, in part, on what she knows about Jose 
and Sara, what might happen to Sara, Jose, or the lesson if she carries out one of the three 
tasks, and what her goals are for her relationships with both students as well as for the whole 
class. Mrs. Jones decides to move forward with the lesson but asks Jose if it is ok to speak after 
class, leaving an opening for him to say he would rather speak now. And she goes to stand next 
to Sara. This type of complex decision-making based partially on prior knowledge and almost 
always with incomplete knowledge, is typical of the decision-making teachers carry out 
hundreds of times a day (Ball, 2018; Lampert, 2003). 


In addition to these decisions, Mrs. Jones made other decisions about how to begin the 
lesson mathematically and then, which substantive aspects of students’ responses to take up 
when they offered their thinking. But she had more decisions to make. For example, once it 
became clear students had questions about the terms in the learning objective, Mrs. Jones 
needed to make decisions. Should she quickly define each term herself? Or should she clarify 
the meaning of each term as a part of her planned instruction on graphing? Should she gather 
more student responses to her original question before having the students define terms? 
Should they verbally define the words or do so in writing, given the English language proficiency 
among her students? There are many decisions for Mrs. Jones to make. 


The teaching Mrs. Jones does in just these five minutes requires KSCs around many 
different aspects of teaching and learning, from using smartboards, to mathematical concepts 
and procedures, to instructional strategies for language learners, to social-emotional supports. 
For Mrs. Jones and for teachers across California, this is teaching — complex, situated work that 
relies on knowledge, judgment, and skill. It may look different across grade levels and subjects, 
but it is professional work that requires the thoughtful integration of teaching practices (Bersin 
& Sandy, 2009). This view of teaching is consistent with current views of teaching and learning 
(Gitomer & Bell, 2016). 
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California’s teacher education learning system must prepare beginners to carry out this 
type of complex, situated action. The CS7TPs serve as the backbone for the learning system, 
specifying six domains of teaching practice that are deliberately “broad and interconnected 
because the professional practice of teaching must be understood comprehensively as a 
complex, dynamic process in which practical and conceptual elements are woven together” 
(CTC, 2009, p. 2). Those domains are 


1. Engaging and Supporting All Students in Learning (Engaging Learners) 

2. Creating and Maintaining Effective Environments for Student Learning (Creating 
Environments) 

3. Understanding and Organizing Subject Matter for Student Learning (Understanding 
Subjects) 

4. Planning Instruction and Designing Learning Experiences for All Students (Planning 
Learning) 

5. Assessing Students for Learning (Assessing Learning) 

6. Developing as a Professional Educator (Developing Professionalism) 

(CTC, 2009) 
The six CSTPs are broad and overlapping; we now turn to the KSCs required across all 
the CSTPs and around which California regulates the teacher education system. 


The Knowledge, Skills, and Capabilities Beginning Teachers Need 


If the integrated view of teaching and learning described by the CSTPs sounds complex, 
it is. Inside of these six domains there are 38 elements that together support accomplished 
teaching practice — teaching which allows all students to learn and develop. The CSTPs 
deliberately describe accomplished teaching practice, not beginning practice; and, the CSTPs 
take a developmental view of teaching proficiency. Therefore, to specify what a beginning 
teacher needs to know and be able to do on her first day as the teacher of record, the CTC has 
established additional standards, including the Teacher Performance Expectations, 
assessments, and requirements. We completed an extensive document review of these 
standards, assessments, and requirements, and find that beginning teachers must have the 
following five types of KSCs according to the guidance conveyed in these documents: 1) basic 
knowledge and skills, 2) content knowledge of specific subjects, 3) the KSCs necessary to enact 
integrated teaching practices, 4) knowledge of English language development, and 5) 
knowledge of reading development. 


A beginning teacher must demonstrate a sufficient level of basic knowledge and skills 
signified by earning a high school diploma as well as a bachelor’s degree. Basic knowledge and 
skills includes knowing how to use common technologies (e.g., e-mail, PowerPoint, Word, etc.) 
as well as knowledge for citizenship (e.g., the history of the U.S. and California, knowledge of 
the constitution and government). Beyond broad bodies of knowledge, the specifics of what a 
beginning teacher learns about a specific topic will vary by college and university. The depth of 
a teacher’s knowledge will also vary because grading standards and expectations vary across 
institutions. 
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Teachers must also have a sufficient level of content knowledge in the subject matter 
they will teach. California’s elementary teachers are certified to teach in any self-containeds 
classroom from PK to adult, although a large proportion teach in the elementary grades 
because that is where self-contained classrooms are frequently located (CTC, 2018a). An 
elementary teacher’s bachelor’s degree indicates she has learned the basic content knowledge 
and skills of a generalist. Secondary teachers are certified to teach in specific subjects — e.g., 
chemistry, mathematics, English, Spanish, music, etc. — in any grade from PK-adult classrooms. 
Like elementary teachers, for a secondary teacher, the bachelor’s degree signifies competency 
in basic knowledge and skills, but it may also provide KSCs in an area closely tied to the 
candidate’s certification area. 


However, recent research suggests that while strong content knowledge in the subject 
area is necessary for teaching, it is not sufficient (Ball, Thames, & Phelps, 2008; Ma, 1999). 
There is specialized content knowledge related to the teaching of the subject that teachers 
have, but subject matter majors tend not to have. Specifically, in a recent study of physics 
majors and physics teachers, researchers found that on a multiple choice test of content 
knowledge and content knowledge for teaching, both physics majors and physics teachers had 
similar levels of physics knowledge, but physics teachers had additional knowledge about how 
to use and represent that knowledge for teaching (laconangelo, Phelps, & Gitomer, 2017; 
Phelps et al., 2017). Appendix A shows a typical item both groups scored well on as well as an 
item physics teachers outperformed physics majors on. 


This specialized knowledge, or content knowledge for teaching (Ball et al., 2008), is just 
one kind of knowledge that falls into the third category of KSCs beginning teachers need on 
their first day. The third category, the KSCs to enact integrated teaching practices, is a large, 
important, and diverse category. Beginning teachers must, for example, be able to manage how 
time is used in a classroom so that students can learn. Teachers also need to know how to 
assess what students know and use that information while they are teaching a lesson and over 
time as they are planning sequences of lessons. Teachers also need to know and be able to 
interact productively with other adults around their professional obligations. For example, they 
must be able to listen to and work with parents in order to support students. They also need to 
understand when and how to work with teaching colleagues with specialized expertise (e.g., 
special education, English language, counseling, etc.). These are just four of the 45 integrated 
teaching practices beginners need to understand and be able to enact on their first day in the 
classroom. These practices are grouped into six Teaching Performance Expectations (TPEs) that 
delineate the specific integrated practices beginners must have their first day of teaching (CTC, 
2016b). These six TPEs were recently revised, updated, and TPPs were required to implement 
them by the 2017-18 school year. Further, a large-scale survey was carried out to determine the 
degree to which they measured important aspects of teaching that beginners must be able to 
carry out (CTC, 2016c; Ford, D’Mello, Paullin, & Thacker, 2016). 


s A self-contained classroom is one in which the group of students are taught many subjects by one teacher. 
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All beginning teachers must also have knowledge of English language development in 
children. The state’s public school student population has almost 2.7 million learners for whom 
English is not the language spoken in their home (CDE, 2017). This is more than 42 percent of 
the public school population (CDE, 2017). Approximately half of those who speak a language 
other than English are currently considered English learners (ELs); in 2016-17, there were 1.3 
million California students (21 percent) who were classified as ELs (CDE, 2017). Thus, California 
Education Code requires all beginning teachers with ELs in their classroom to know how 
children develop English language proficiency and be able to use that knowledge in their 
teaching practice. There are different levels of knowledge required, depending on the learning 
needs of the students. The lowest level — knowledge of instruction for English language 
development — has been integrated into initial single- and multiple-subject credentialing 
programs since 2002 (CTC, 2015b). Frequently there are higher levels of knowledge needed to 
teach ELs, such as knowledge and skill to provide “specially designed academic instruction 
delivered in English” and “instruction for primary language development and content 
instruction delivered in the primary language” (CTC, 2015b, p. 1). These levels of knowledge 
require additional optional authorizations. 


Finally, the last category of KSCs beginning elementary teachers must have surrounds 
literacy. Literacy is a critical skill in the 21st century, allowing citizens to engage their 
democracy, solve problems, and contribute to the knowledge economy (Lievesley & Motivans, 
2002; UNESCO, 2005). In school and in life, being a proficient reader allows one to learn other 
subjects (Mather, Bos, & Babur, 2001) and engage in our democracy. For elementary teachers, 
no matter what grade level is being taught, teachers must have enough knowledge and skill 
around teaching reading that students with a wide variety of reading levels can access the 
subject matter being taught. Therefore, all beginning elementary teachers must have 
specialized knowledge of reading development (CTC, 2018b; 2018c) focusing on how students 
learn to read and how to teach students to read (Mather et al., 2001; McCombes-Tolis & Feinn, 
2008). 


Together, these five categories broadly describe what teachers need to know and be 
able to do on their first day. Teachers must have an appropriate level of basic knowledge and 
skills, content knowledge, integrated teaching practices, knowledge of English language 
development, and knowledge of reading development. California regulates and monitors the 
pipeline around these categories. It is worth noticing that four of the five categories focus more 
on knowledge than capabilities. This focus on knowledge is a well-documented and oft- 
maligned aspect of teacher pipelines (Gitomer & Zisk, 2015; Goldhaber, 2007; Mitchell, 
Robinson, Plake, & Knowles, 2001; Wilson, 2009). 


A common issue many states face is how best to prepare general education teachers for 
supporting students with special needs. Prior to 2016, preservice teachers working toward a 
single- or multiple-subject credential in California were not required to demonstrate KSCs 
around teaching students with special needs, nor were there requirements for general 
education teachers to learn specific evidence-based practices common in special education 
(California’s Statewide Special Education Taskforce, 2015). However, with the adoption of 
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revised TPEs (CTC, 2016b) in 2016, the state changed this. Multiple- and single-subject 
beginning teachers must show proficiency on a common “core” or “trunk” of preparation 
experiences that cut across both special and general education practices (California’s Statewide 
Special Education Taskforce, 2015). Two recent meetings in December 2017 and February 2018 
have specified these new requirements and identified how general and special education 
teachers might share a common core of KSAs (CTC, 2017h; CTC, 2018e) and TPPs are required 
to have aligned their programs to the new TPEs by the end of the 2017-2018 school year. 
Because some programs will have complied with the TPE reform very recently, it will likely take 
some time to see any changes in general education teachers’ proficiency. 


Pipeline Institutions and Policies 


Where and when do novices learn the five categories of KSCs required of them? As 
shown in Figure 1, K-12 schools, community colleges, four-year institutions of higher education 
(IHEs), California workplaces, and TPPs all provide prospective teachers important learning 
opportunities related to the KSCs required by the CSTPs. 


There are many paths through which a teacher may travel prior to their first day in front 
of a classroom of students. Some students will take a direct route in which they graduate from 
high school, graduate with a bachelor’s degree from a 4-year IHE, complete a TPP at either an 
institution of higher education or local education agency and receive a preliminary credential, 
obtain employment in a California public school, complete an induction program and obtain a 
clear credential. This route proceeds straight down the right side of Figure 1. Other students 
may attend community college for one or two years, transfer in to a 4-year IHE, and then follow 
the same route as those who went directly to a 4-year IHE. Still others will go directly into the 
workforce and then return to a community college and/or 4-year IHE to pursue a teaching 
career.9 And finally, others will proceed partially down one of these paths, obtain employment 
in a California public school, and continue to make progress toward their teaching credential 
while teaching students in California’s K-12 classrooms. There is diversity in the paths 
prospective teachers may take through the pipeline. 


We first describe the general roles the institutions in Figure 1 play in developing 
teachers’ KSCs and then provide a high-level overview of the main policies, regulations, and 
assessments that structure the learning opportunities provided by pipeline institutions. We 
then detail some of the main pathways through the teacher education system, noting the 
relative number of novices moving through each. In the next section, we suggest criteria that 
might be used to evaluate the efficacy of the teacher education system and then evaluate it 
against three of those criteria. 


9 California TPPs have recently been expanding a new pathway called “blended programs” that integrates a 
bachelor’s degree with the requirements for a teaching certificate. In 2015, the state budget included $10 million 
in planning grants of $250,000 each for colleges to better design these programs 
(https://edsource.org/2017/undergraduate-education-major-banned-for-56-years-returns/585830). The programs 
are new and therefore there is little publicly available information about these programs or their efficacy. 
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Main Teacher Education Institutions and Roles 


K-12 schools. K-12 public schools play a critically important and under-recognized role in 
the provision of preservice teachers’ learning opportunities at the beginning, middle, and end 
of the teacher education pipeline. At the beginning of the pipeline, K-12 schools are the 
institutions that provide opportunities for aspiring teachers to develop the KSCs necessary to 
teach effectively. In particular, the basic knowledge and skills as well as the subject-specific 
content knowledge that a teacher eventually brings to her own classroom reflects the 
knowledge that she gained from the K-12 schools she herself attended. 


In the middle of the pipeline, K-12 schools provide the practice-based settings in which 
teachers enrolled in a TPP practice integrated teaching practices as well as apply their content 
knowledge under the guidance of IHE faculty/staff members and K-12 teacher mentors. As has 
been written about extensively, the classrooms in which candidates work with K-12 students 
provide critical learning experiences for beginning teachers (e.g., student teaching) (Grossman, 
2010; Grossman, Hammerness, & McDonald, 2009; Hollins, 2015). And despite varied models to 
support the collaboration of TPPs and K-12 professionals, those learning experiences are 
profoundly shaped and often controlled by K-12 professionals. This may be especially true for 
beginners participating in intern programs in which they are “student teaching” in the form of 
being a teacher of record. 


Finally, at the end of the teacher education pipeline, K-12 schools provide newly 
certified teachers with the mentors and induction program curricula necessary to complete the 
induction phase of teacher preparation, and further develop their capability in carrying out 
integrated teaching practices. Of course, K-12 schools are also a beginner’s workplace, which 
can provide varied learning opportunities, ranging from more to less productive. Given this 
frequent involvement in three locations on the teacher education pipeline and the nature of 
the learning opportunities provided in these locations, K-12 schools are a very important 
institution participating in the preparation of beginning teachers. 
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Figure 1. Institutions in the teacher education learning system 
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Community colleges and 4-year IHEs. Community colleges and 4-year IHEs provide 
additional learning opportunities for aspiring teachers to further develop their basic knowledge 
and skills; they also provide teachers their primary learning opportunities to learn more 
advanced content knowledge. For example, a teacher seeking a single-subject certification in 
mathematics, English, history, etc. will likely deepen the content knowledge initially developed 
in her K-12 years in community colleges and 4-year IHEs. These institutions also provide aspiring 
multiple-subject teachers an extended opportunity to develop the broad content knowledge 
for teaching elementary-aged students. To varying degrees, these institutions will provide 
additional learning opportunities around other important KSCs, such as knowledge and 
proficiency in a second language, capabilities of working with other adults, and general 
knowledge of national and global social, economic, and political issues. All California teachers 
are required to have a bachelor’s degree, therefore, all beginning teachers rely on 2- and 4-year 
IHEs to provide important learning opportunities. 


California’s workplaces. Many beginning teachers will have interacted with California’s 
workplaces and the attendant learning opportunities they provide. These workplaces provide 
learning opportunities that are necessarily specific to the particular business or industry of that 
workplace. Some prospective teachers may have opportunities to learn interpersonal skills from 
working at fast food restaurants or grocery stores. Such skills might be useful for learning how 
to work with other school professionals or students’ families. Other prospective teachers will 
have the opportunity to learn how students understand content from tutoring K-12 students 
for test preparation companies, tutoring through neighborhood social networks, teaching in a 
private school or early childhood center, or even teaching in a public school on an emergency 
permit. Still others will have the opportunity to learn KSCs through careers such as banking, IT, 
engineering, and marketing. The quality and nature of the learning opportunities provided in 
California’s workplaces vary dramatically and may or may not be important to the ultimate KSCs 
a beginning teacher has on day one. These workplace learning opportunities can occur 
anywhere along the teacher pipeline, beginning when the aspiring teacher is in high school all 
the way through to when the teacher has her own classroom. Such learning opportunities are 
critical to some of California’s alternative certification or intern pathways. 


Teacher preparation programs. Among the preliminary single- or multiple-subject 
teaching credentials issued in California, the majority (63 percent) are awarded to candidates 
that completed a post-baccalaureate TPP (referred to as a “traditional” program). As shown in 
Table 1, in 2016-17, 9,742 California teachers received a multiple- or single-subject preliminary 
credential through a post-baccalaureate TPP in the state of California. This represents just over 
72 percent of all newly single- and multiple-subject preliminary credentialed California 
teachers. When broken down by preliminary credential type, 70 percent of single-subject 
credentialed teachers and 74 percent of multiple-subject credentialed teachers were 
credentialed through a California-accredited post-baccalaureate TPP in 2016-17. The remaining 
newly preliminary credentialed single- and multiple-subject teachers, respectively, were 
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credentialed through TPPs outside of California or in another state, another country, or a non- 
specified entity. 


TPPs provide critical, and in many cases the only, opportunity to learn the specialized 
KSCs teachers need for teaching: integrated teaching practices, knowledge of English language 
development, and knowledge of reading development. TPPs collaborate intensively with K-12 
professionals in the provision of these learning opportunities, shown in Figure 1 as the K-12 
school providing the practice-based context for preservice teacher learning. 


Assessments and Requirements 


California regulates these institutions that make up the teacher education pipeline and 
the learning opportunities they provide through varied assessments and requirements. To 
simplify, we focus on the main assessments and requirements, pointing the reader to reports 
and websites for additional information. 


Accreditation of pipeline institutions. Accreditation is a process used across many 
public sectors to ensure both quality and standardization of services. Police departments, 
medical schools, and hospitals all use accreditation to ensure that their respective professional 
standards are implemented by practitioners. The same is true in the teacher pipeline; in 
particular, 4-year IHEs, TPPs,10 and induction programs all must be accredited. Accreditation of 
K-12 public schools is not required; however, all K-12 public schools must adhere to the 
California Department of Education’s (CDE) requirements as well as state administrative codes 
and statutes. This leaves only California workplaces as unaccredited, although some workplaces 
are accredited within their own fields. State law requires aspiring teachers to obtain a 
bachelor’s degree from a regionally accredited IHE. Accreditation requirements therefore shape 
the KSCs that students develop while pursuing a bachelor’s degree. For example, Western 
Association of Schools and Colleges (WASC), the primary regional accreditation agency for 
colleges and universities in the state of California, requires all accredited IHEs to (a) “define and 
incorporate into all of its undergraduate degree programs a substantial component of general 
education designed to ensure basic collegiate skills, breadth of knowledge, and the structures 
of intellectual inquiry”; and (b) design “educational objectives for the general education 
program that include demonstrated competence in writing, critical thinking, scientific literacy, 
computational skills, and an introduction to the broad domains of knowledge” (WASC, 2015, p. 
28). 


TPPs are accredited by the CTC and only accredited programs can recommend teacher 
candidates to be licensed.11 The CTC has recently revised its TPP accreditation policy but these 
revisions are just beginning to impact the teacher education system. Program accreditation 


10 There is one exception to this. The Teacher Preparation Pipeline program, established in 2006-07 to provide 
more certified STEM and career and technical education teachers, is not accredited by CTC. 

11 National accreditation can be substituted for state accreditation as stipulated by Education Code 44374 (f). 
However, substitution requires alignment studies of both standards and accreditation activities, a program review, 
and a site visit (CTC, 2018a). This means that from a practical perspective, the CTC still reviews and accredits 
preparation programs. 
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takes places in seven-year cycles, with annual reporting requirements, document reviews in 
each of four years, and one site visit in a sixth year. Programs must meet six standards as well 
as subject-specific pedagogical standards (CTC, 2017b). Standards focus on the curriculum of 
TPPs, how well candidates are prepared for the TPEs, clinical practice, monitoring candidates 
over time, implementing a Teaching Performance Assessment (TPA) (CTC, 2015a), and providing 
each candidate with an individual learning plan (ILP) that is used during the induction phase of 
the pipeline. 


Finally, induction programs — state mandated programs that continue a beginning 
teacher’s learning from the time a preliminary credential is issued to the time a clear credential 
is issued — must also be accredited by the CTC. Accreditation occurs around six standards 
including the program’s purpose, the components of the mentoring design, the implementation 
of well-designed ILPs, the quality of mentors, determining candidate competence for the 
purposes of recommending a clear license, and program responsibilities for assuring program 
quality (CTC, 2017a). Accreditation also requires specific induction program features such as the 
length of the program and frequency of mentoring, among others. 


Institutional requirements. In addition to accreditation requirements, pipeline 
institutions specify varied institutional requirements that provide learning opportunities for 
beginning teachers. For example, IHEs determine how many hours of coursework are required 
for a specific major as well as how the hours are distributed across subtopics within the major. 
IHEs can also specify minimum GPA requirements for certain majors or for entry into the 
university. TPPs, through their admissions requirements, might specify that candidates applying 
to the program must have completed a certain number of hours working with school-aged 
children or they may specify a minimum GPA. Two-year and 4-year IHEs, as well as TPPs have 
the freedom to specify course requirements and assessments, both of which play a crucial role 
in shaping KSCs. 


Assessments. California has assessments or substitutions for those assessments in each 
of the previously discussed five categories that broadly describe what teachers need to know 
and be able to do on their first day. The most commonly completed assessments for preservice 
teachers in traditional and intern pathways are shown in Figure 2. Appendix B lists the 
requirements and substitutions exhaustively and shows differences between traditional and 
intern pathways to certification. In order to meet the basic knowledge and skills requirement, 
preservice teachers can either take and pass the California Basic Educational Skills Test (CBEST) 
or score high enough on one of a few other assessments (SAT, ACT, or AP). To meet the content 
knowledge requirement, beginners must complete a CTC-approved subject matter program or 
take and pass the California Subject Examination for Teachers (CSET). The content knowledge 
requirement must be met before an aspiring teacher can begin student teaching, although 
some TPPs may encourage students to pass the exam before entry into the program. Once 
enrolled in a TPP, preservice teachers complete integrated coursework and practice-based 
placements that support the candidates’ knowledge of English language development. 
Candidates also learn about and practice integrated teaching practices. Candidates’ proficiency 
on some of these teaching practices are assessed by the Teaching Performance Assessment 
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(TPA) used by the program. Finally, candidates seeking a multiple-subject credential must 
demonstrate knowledge of reading development by passing the Reading Instruction 
Competence Assessment (RICA) prior to program completion. The last of these assessments — 
the TPA and RICA — must be passed before the TPP recommends the candidate for a credential. 


Graduated licensure policy. California has a graduated licensure policy that treats 
California teachers as continuous learners. Specifically, a prospective teacher must first 
successfully complete the preliminary certificate requirements that are set by the preparation 
program (shown in red in Figure 2) as well as the CTC assessment requirements (shown in green 
and blue in Figure 2). After the preservice requirements are complete, a candidate is 
recommended to the CTC for certification and granted a preliminary certificate. In the spirit of 
supporting continuous novice learning, the novice teacher then has five years to complete an 
accredited induction program (red outline in the K-12 school in the lower right corner of Figure 
2) as well as the successful completion of the ILP. Once both of these requirements are met, the 
teacher is eligible for a clear credential. 


Induction policy. In contrast to many states, California has a formal induction program 
that is regulated by the CTC and implemented by roughly 160 institutions, including 
universities, districts, charter management organizations, and county offices of education. In 
order to earn a clear teaching credential (i.e., one for which there are no further academic 
requirements), all beginning teachers must complete a two-year accredited induction program 
(CTC, 2017a). Induction programs pair beginners with appropriate mentors to receive an 
average of one hour per week of mentoring. Before leaving a California TPP, a beginning 
teacher develops an ILP that is used as the basis of the work mentors and beginners do 
together. All ILPs must align to the 6 TPEs and over the two years of the program, the beginner 
must show demonstrated proficiency in all six CSTPs standards, which guide in-service teachers’ 
growth. 
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There are multiple pathways through Figure 1, as noted previously. Many teachers 
move down the right side of the figure from high school through a post-baccalaureate TPP to 


induction and a clear credential. But 


many others have pathways that skip over preparation 
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programs as we tend to think of them. For example, an individual might earn a bachelor’s 
degree and then go work for a district full-time as a paraprofessional and then enter a district- 
based intern program that results in a preliminary certificate, which then leads, through 
induction, to a clear credential. 


Pathways that eventually move through traditional TPPs may or may not be better than 
other pathways, but they do carry state-designed assurances of first-day preparedness. Intern 
pathways, short-term staff permits, provisional internship permits, or waivers do not provide 
citizens the same assurances. As such, novices working on waivers, permits, and intern 
credentials are often referred to as teachers working on “substandard” credentials. 


In the case of the intern pathway, intern beginning teachers are concurrently teaching 
children and completing the necessary coursework and requirements to earn either a multiple- 
or single-subject preliminary teaching credential. This is a significant and growing group of 
teachers: 16 percent (2,566 teachers) of preservice teachers were working toward their 
multiple- or single-subject teaching credential through a California-based intern program in 
2016-17. These individuals with an intern teaching credential (a) possess at least a bachelor’s 
degree from a regionally accredited institution of higher education; (b) met the basic skills 
requirement, unless exempt by statute or regulation; (c) demonstrated knowledge of the 
Constitution of the United States; (d) demonstrated subject matter knowledge; and (e) are 
enrolled in a CTC-approved TPP. As of their first day of teaching, these individuals with intern 
credentials must have completed intern preservice preparation provided by a CTC-approved 
program sponsor, and received a recommendation from a CTC-approved intern program. Prior 
to teachers assuming daily teaching responsibilities, a signed Memorandum of Understanding 
must be in place between the CTC-approved program sponsor and the school district employing 
the teacher with the intern teaching credentialing, detailing the support and supervision that 
will be provided to the individual with an intern teaching credential. Additionally, prior to 
assuming daily teaching responsibilities, a teacher with an intern teaching credential must be 
assigned a mentor teacher. A minimum of 144 hours of support/mentoring and supervision 
must be provided to each intern teacher per school year with 45 hours of additional support 
and supervision focused on supporting English learners (Intern Teaching Credentials, 2014). 


Two other ways a novice might begin teaching a group of students without the state- 
designed assurances result from the need to fill a temporary staffing vacancy or need. In 2016- 
17, another 38 percent (5,061) of beginning teachers had a permit for these purposes. Such 
permits, which are issued for no more than one year, may be a short-term staff permit or a 
provisional internship permit. To obtain either, an individual must (a) hold at least a bachelor’s 
degree from a regionally accredited college or university; (b) meet the basic skills requirement, 
unless exempt by statutes or regulations; (c) successfully complete specific courseworki2 with a 


12 For the single-subject, short-term staff permit, at least 18 semester units of course work in the subject to be 
listed. For the multiple-subject, short-term staff permit, at least 10 semester units of course work in each of at 
least four of the subject areas (i.e., language studies, history, literature, humanities, mathematics, the arts, 
science, physical education, social science and human development) or at least 10 semester units of course work in 
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grade of “C” or higher, “Pass,” or “Credit”; and (d) complete the background check process 
(CTC, n.d.a). In contrast to the traditional pathway, these permits allow novices to work with 
students, without having fully met the subject matter requirement, the reading or EL 
requirements, nor having demonstrated proficiency on integrated teaching practice. While 
current state data does not allow us to compare the effectiveness of various pathways, at a 
minimum we know that these novices have not had the learning opportunities nor met the 
proficiency standards the state has designed to ensure novices are adequately prepared. 


A final category of preservice teachers is those with waivers. In 2016-17, 2 percent 
(234) of preservice teachers in California classrooms had a waiver. A waiver means the 
candidate (a) holds a teaching credential but is teaching outside of their credentialed 
authorization(s), (b) has been permitted additional time to complete a credential requirement, 
or (c) has been placed in a teaching position, potentially without a credential, when all other 
hiring efforts have been exhausted by the employing agency (CTC, n.d.d). More specifically, 
there are two types of waivers: short term and variable term. Short-term waivers may be 
approved at the local level to provide the district with one semester or less to address 
unanticipated, immediate, short-term organizational needs by assigning only individuals who 
hold basic teaching credentials to teach outside their credentialed authorizations, with the 
consent of the teacher. Short-term waivers may be issued once to any individual teacher and 
only once for a given class. Variable-term waivers are reviewed by CTC staff and acted upon by 
the CTC at a regularly scheduled meeting. They provide the district up to one year for a specific 
period of time set by the CTC to: (a) allow individuals additional time to complete a credential 
requirement, (b) facilitate assignment in school programs addressing issues of educational 
reform, (c) allow geographically isolated regions with severely limited ability to develop 
personnel time to hire and develop personnel, or (d) obtain waivers for situations when all 
other hiring efforts have been exhausted (CTC, n.d.d). 


It is clear that the number of teachers entering teaching with substandard credentials 
has grown markedly from 2012-13 to 2016-17. Table 1 describes patterns in the credentials of 
teachers entering California schools over time. In 2016-17, over one-third of newly single- and 
multiple-subject certified California teachers had either not met all of California’s teacher 
credentialing requirements or were not credentialed in the subject area that they were 
teaching but were permitted to teach in California K-12 classrooms through waivers, permits, or 
as interns (CTC, n.d.d). Second, 17 percent of newly multiple- or single-subject certified 
California teachers were prepared by out-of-state or out-of-country programs for which 
California cannot regulate the experiences of preservice teachers (CTC, n.d.c). However, 
California does regulate the credentialing of these teachers upon entering California. 


For an out-of-state prepared teacher to qualify for a California teaching credential, they 
must have a bachelor’s degree and an out-of-state teaching license; however, they are eligible 
to receive initial certification in California without having completed an out-of-state exam that 


each of three subject areas and an additional 10 semester units of course work in a combination of two of the 
remaining subject areas. 
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meets the requirements of the California Basic Skills Examination (CTC, n.d.b). Out-of-state 
prepared teachers have up to one year after submission of their application to complete the 
California Basic Skills Requirement (CTC, n.d.b). These two groups of teachers — those teaching 
on substandard credentials and out-of-state teachers — made up over half of California’s “new” 
teacher workforce in 2016-17 (see Figure 3). These teachers may have either different learning 
opportunities in teacher preparation programs not regulated by California or not yet engaged in 
the full range of learning opportunities that California policies deem to be important. This 
reality further emphasizes that learning opportunities in pathways differ and to have a full 
understanding of California’s teacher education system, one must trace those pathways and 


the learning opportunities inherent in them. 
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Criteria to Judge the Teacher Education Pipeline 


Having described what beginning teachers are expected to know and be able to do as 
well as the institutions, assessments, and regulations that guide and ensure such learning, we 
now turn to a consideration of the quality of that learning system. Drawing from decades of 
research in education policy, we suggest five broad criteria that can be used to answer the 
research question “How should we judge the quality of California’s teacher education system?” 
Those criteria are the degree to which the pipeline 1) has aligned policy messages for 
stakeholders, 2) prepares capable novice teachers, 3) generates relevant data, 4) prepares 
enough teachers, and 5) is cost-effective. In this section, we describe these broad criteria. 
Subsequently, we evaluate the system on the first three criteria. The issue of teacher supply is 
addressed in a series of recent reports (Carver-Thomas & Darling-Hammond, 2017; Darling- 
Hammond, Furger, Shields, & Sutcher, 2016) as well as a Getting Down to Facts II paper 
(Darling-Hammond & Sutcher, 2018), and the under-researched and exceedingly complex issue 
of the system’s cost-effectiveness is beyond the scope of this already lengthy paper. 


Aligned Policies 


For the better part of the last 30 years, education research emphasizes the importance 
of alignment and coherence in any system that is designed to improve teaching and learning 
(e.g., Coburn, Hill, & Spillane, 2016; Cohen & Hill, 2001; McLaughlin, 1987; Penuel et al., 2010; 
Spillane, 2006). Without alignment of policies, curricula, assessments, and other policy tools, 
one is likely to have an incoherent system, prone to uneven outcomes and ongoing stakeholder 
dissatisfaction. There are many ways for a system to be misaligned. Teaching standards may be 
out of date with new student learning standards or teacher tests might measure static content 
knowledge, not the situated and dynamic content knowledge for teaching beginners need on 
their first day of teaching. This suggests that a necessary, but not sufficient condition for an 
effective teacher education system is an aligned and coherent system. 


Capable Teachers 


Any teacher pipeline must produce teachers who are ready to begin teaching on their 
first day in the classroom (Ball & Forzani, 2009). They must be able to carry out common tasks — 
correctly interpreting what students know and do not know, explaining subject matter, and 
discussing students’ progress with their parents or guardians. While beginning teachers are just 
that — beginners — the teacher education pipeline must prepare them enough to be safe for all 
children on their first days as the teacher of record. The idea that beginners must be “safe to 
practice” comes from a long history in the licensure of professionals that suggests the state’s 
role is to ensure that a person charged with carrying out a professional task will do so safely. It 
is not the state’s role to judge whether a beginning professional is excellent or even effective. 
Instead the state’s role is to protect the public by ensuring safety. 


This distinction is easily understood in the restaurant industry. Through regulations 
around food storage, food preparation, hand washing and the like, the state ensures that 
restaurants will serve food that will not make the public sick. The state does not ensure the 
food will be delicious, or even good. Those criteria are left to professional and economic forces. 
The state protects the public from food that endangers their health; it ensures the food is safe. 
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The same is true in law and medicine. A medical or law licensure does not ensure a high-quality 
lawyer or doctor. Instead, the state ensures competent lawyers and doctors, both of whom 
know important knowledge and are proficient in their respective fields. In teaching, the state’s 
role is to ensure that on their first day, teachers are ready to teach all children and will not 
inflict educational harm on them. 


But how does the state know if a beginner teacher is safe? This requires judgment based 
on expert understandings of the KSCs necessary for beginner teachers as well as knowledge of 
the specific students the teacher will encounter on her first day. California, like other states, 
regulates what teachers must know and be able to do. In most states, there is a heavy 
regulatory emphasis on what teachers know as measured by multiple-choice teacher tests 
(Gitomer, Brown, & Bonnett, 2011; Gitomer & Qi, 2010). Increasingly there are standardized 
requirements compelling candidates to demonstrate their capabilities through performance- 
based assessments administered during their TPPs (e.g., Goldhaber, Cowan, & Theobald, 2017; 
Henry et al., 2013). To guarantee teachers learn what they need to across the pipeline, the 
assessments and regulations used by the state must ensure all teachers know and can carry out 
enough teaching practices in their subject areas so that children are protected from unsafe 
practice. Or in the words of the Teacher Performance Expectations, “establish, maintain, and 
monitor inclusive learning environments that are physically, mentally, intellectually, and 
emotionally healthy and safe to enable all students to learn...” (TPE 2, element 3, CTC, 2016b). 


If we accept these premises, then programs, and thereby the state, must have a body of 
standardized, detailed, relevant, performance information upon which to judge a candidate’s 
mastery of the relevant knowledge and integrated teaching practices necessary to teach on the 
first day of school. Such information can be used both summatively and formatively to improve 
programs and identify any programs that are producing candidates deemed unsafe to practice. 
The state can evaluate the quality of these regulations and assessments by considering how 
performance on the state’s preservice requirements and assessments are related to beginners’ 
teaching practice. They can also do so by seeking out other validity evidence that creates a 
strong argument for the reasonableness of the assessment being used (Kane, 2013). Not all 
assessments or requirements will be predictive of beginners’ eventual teaching practices, but 
wise policy should seek to understand the relationships between preservice requirements and 
assessments and those eventual teaching practices. 


Relevant Data 


Evidence-based policymaking and improvement processes require data that can help 
diagnose problems and inform appropriate action (Bryk et al., 2015). A teacher education 
pipeline data system must provide critical information that is linked to the outcomes the 
system values. These valued outcomes may vary — perhaps at one time, there is a need for 
more mathematics teachers and at another time, the focus is on credentialing teachers with 
stronger skills in serving bilingual students or teachers more likely to stay in the profession. One 
system may value teachers with specific social-emotional skills or specific content knowledge. 
Irrespective of the specific valued system outcome(s), the system’s data must provide critical 
disaggregated information known to be related to teaching quality. Critical information is data 
logically related to valued outcomes of the system. If, for example, a system values the number 
of bilingual teachers in the public schools, it will not only provide data on how many bilingual 
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credentials are completed; it will also trace the progress of subgroups of individuals most likely 
to earn bilingual credentials, which requires disaggregated information — information provided 
at the proper level of detail such that it can inform improvement efforts and policymaking. To 
continue with the bilingual teachers’ example, it is helpful to know the number of bilingual 
candidates that earn a preliminary credential, but it would be even more helpful to know what 
pathways and programs those bilingual candidates came through in order to determine 
whether and how those pathways should be strengthened or incentivized in some way. It 
would also help to know how many of those credentialed bilingual candidates took jobs in 
bilingual placements versus taking more general teaching placements and how long they stayed 
in those placements. Such critical, disaggregated data across the teacher pipeline is necessary 
to support both system improvement and policy goals. 


This type of investigation — tracing the progress of specific groups through a pipeline — 
also requires linked data. For example, linked data in this case might be data that traces back to 
the community college and IHE populations to determine who was bilingual, how many of 
those individuals were interested in education and which ones eventually completed a teaching 
certificiate and took a job. Without that critical information, it is impossible for the state, 
community colleges, IHEs, and TPPs to know how best to increase the number and quality of 
bilingually certified teachers. Linked data can inform improvement efforts by allowing TPPs to 
understand students’ incoming knowledge and skills, their progress through various majors and 
certification programs, and their eventual employment in K-12 classrooms. Linked accessible 
data are required by the national teacher preparation accrediting body, the Council for the 
Accreditation of Educator Preparation (CAEP, 2016).13 They are also the subject of national 
reports by advocacy and scholarly organizations (Deans for Impact, 2016; Feuer, Floden, 
Chudowsky, & Ahn, 2013) as well as the target of foundation-based reform efforts (Carnegie 
Corporation of New York, 2001). Such data must, however, protect both individual’s 
confidentiality and pipeline institutions’ independence if they are to be used to guide 
improvement efforts. 


Enough Teachers 


The state is responsible for providing free education to California’s children (CA. Const. 
art. IX, § 5). To the degree that this constitutionally mandated responsibility is unfulfilled 
because there are not enough teachers for the state’s classrooms, the state must act. This 
means that the state has a role to play in monitoring and ensuring the adequacy of the supply 
of teachers across the pipeline and over time. The event of not having enough teachers is of 
particular concern. In California, like many other states, there are specific subjects and 
geographic locations that regularly suffer from teacher shortages (Carver-Thomas & Darling- 
Hammond, 2017; Darling-Hammond, Furger, Shields, & Sutcher, 2016; Darling-Hammond & 
Sutcher, 2018). To the degree that there are chronic teacher shortages that adversely impact 
groups of students’ educational opportunities (e.g., African American students, students in rural 
districts, etc.), legal scholarship suggests the state is not fulfilling its duty to provide a “free” 


13 CAEP standards 4 and 5 require TPPs gather data on the effectiveness of program completers and the 
satisfaction of completers and employers, as a part of each program’s continuous improvement efforts that make 
use of multiple valid and reliable measures. 
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and adequate public education as guaranteed by the California constitution and subsequent 
court rulings (Gordon, 2016). 


The event of preparing too many teachers may be of some, albeit less, concern. There is 
an implicit social contract between the state and citizens preparing to be teachers. That 
contract implicitly suggests that the state will not prepare professionals for jobs that do not 
exist. This concern, however, is not usually a large concern because of specific shortages, 
turnover, and geography. In addition to chronic shortages (Darling-Hammond et al., 2016), 
there are a significant number of teachers who leave teaching every year, making those jobs 
available for beginners. Frequently, there are also nearby geographic areas or states that are 
able to employ teachers who cannot find a job close to home. Thus, as long as the social 
contract is generally upheld, the event of the teacher pipeline producing too many teachers is 
less of a concern than producing too few teachers. 


Cost-Effective 


A teacher pipeline system should be cost-effective for both the candidates seeking to 
become teachers and the taxpayers supporting the development of a teacher pipeline. From a 
candidate’s perspective, the cost of certification should be reasonable in terms of dollars and 
the time necessary to become certified. Cost-effectiveness is a notoriously challenging criterion 
to fully quantify, but there is precedent for doing so in other areas of education (e.g., Denton & 
Smith, 1985; Borman & Hewes, 2002; Knight, 2013). Citizens also bear the cost of preparing 
teachers because their tax dollars are used to provide support to public IHEs in their own states 
and through the federal government for subsidized loans for higher education. Thus, it is 
helpful to understand the degree to which the provision of teacher education is cost-effective 
for the teacher workforce and the economy more generally. Such information must be put into 
the context of similar human improvement professions that serve the common good — nurses, 
civil servants, psychologists — in order to help citizens and policymakers make sense of any cost- 
effectiveness estimates. 


Given these five criteria by which a teacher education system might be judged, we now 
turn to an assessment of the first three of those five criteria: a system that is aligned, produces 
capable teachers, and generates relevant data for improvement purposes. We focus on these 
three criteria because another Getting Down to Facts II paper takes up the question of whether 
the system produces enough teachers (Darling-Hammond & Sutcher, 2018), and a fair 
treatment of the system’s cost-effectiveness requires its own report. 


System Alignment 


To investigate the alignment of California’s teacher education system, we carried out an 
extensive document analysis and review of publicly available data. We find that California’s 
teacher education system, regulations, assessments, and policies are aligned at a high level 
around the CSTPs. However, this high-level alignment masks variation in ground-level 
implementation. This ground-level variation may mean that candidates have varying 
opportunities to learn and practice important KSCs, however, there is little data available to 
determine whether and how this variation might influence the preparedness of beginners. The 
document analysis also suggests that there is strong standardization around the specific 
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knowledge candidates are responsible for prior to entering the classroom. There is less 
specification and standardization around the specific integrated teaching practices candidates 
are required to demonstrate. 


High-Level Policy Alignment 


Looking across the institution and policy descriptions of the teacher education system 
detailed previously, the most important source of high-level alignment comes from the CSTPs. 
California’s teaching standards are used in institutions before and after preliminary certification 
to orient the regulations, assessments, and policies toward common standards. This creates 
alignment in the system because all of the policies and assessments must refer back to this 
single set of six standards. For example, induction and preparation programs are required to 
link the learning opportunities they offer candidates back to the CSTPs through the TPEs (CTC, 
2016d; 2016e). 


Looking more deeply at specific requirements, there is also high-level alignment in the 
system, created by similar organizational structures and a common substantive focus on the 
same areas of teaching. This high-level alignment is accomplished by requiring that important 
regulations and assessments be organized into the six TPEs that specify what beginning 
teachers should know and be able to do. Specifically, the system is aligned across the preservice 
and induction phases of teacher education through the alignment of TPP accreditation 
regulations, the TPA required by the end of candidates’ program, accreditation regulations for 
induction programs, and successful progress on an ILP prior to earning a clear credential. This 
alignment is visible in Figure 4, a figure modified from other similar figures found in TPP 
documentation. Reading horizontally across Figure 4, the colors substantively show how the six 
TPEs (and 46 elements) used by preparation programs to guide their curriculum and 
accreditation are grouped into six domains that are aligned with the six CSTPs and the CSTPs’ 
associated 38 elements. The left side of the figure shows the six TPEs and elements that are a 
part of preservice accreditation and program design. The right side shows the six teaching 
standards and elements that guide induction accreditation and design. In the middle of the 
figure, one TPA, the CalTPA, was selected as an example. The CalTPA is also aligned to the TPEs, 
and by extension the CSTPs. The other TPAs are also similarly aligned. The elements marked in 
the center of the diagram are taken from the four types of rubrics in the revised CalTPA. 


The high-level alignment of the system is clear reading across any horizontal color panel. 
For example, one CS7P that guides induction policy and assessment is Understanding and 
organizing subject matter for student learning, shown in the third horizontal panel in tan. This 
CSTP is aligned with the related preservice TPE of the same name. Despite two additional 
elements in the TPE domain, the standards are very similar — both focus on how teachers 
understand and use subject matter to support student learning. High-level alignment is further 
strengthened because this standard is measured in three of the four TPA tasks that are a part of 
the CalTPA. 


We acknowledge, however, that there are many teaching practices that might 
demonstrate a beginner has reached a certain level of proficiency in making subject matter 
comprehensible to students. For example, a program might require candidates to model and 
explain how to write a five-paragraph essay, actually requiring candidates to give a brief lecture 
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in which the candidate generates written text in front of the class, narrating and modeling how 
she is thinking about writing the essay as she goes. This approach would align with TPE 1: 
engaging and supporting all students in learning in the preservice program and could be 
evidence of preservice teacher competency. During induction, however, a mentor might see 
the same person carrying out a different brief lecture in which the teacher might compare two 
essays that use different approaches to gain the attention of the reader in the first paragraph. 
The teacher’s explanation of how the two essays compare to one another could also show the 
teacher’s proficiency with understanding and organizing subject matter for student learning. 
Being able to write an essay in front of students while thinking aloud is a different practice than 
comparing two essays’ opening paragraphs. Both would align at the level of the TPEs and the 
CSTPs, but they are different teaching practices. 


The high-level alignment and coherence this analysis finds in the teacher education 
system is an important asset to the state’s efforts to create an effective teacher education 
pipeline. The accomplishment of creating such a system should not be underestimated. 
However, given the relatively low level of student performance in California compared to other 
similar states (NCES, 2015, 2017), it is necessary to analyze what teachers are required to know 
and be able to do at a finer grainsize. It is this analysis to which we turn now. 


Standardization and Variation in Knowledge Requirements 


Within each of the pipeline institutions that influence teacher development (depicted in 
Figure 2) there are policies and procedures that shape the KSCs that teachers gain in California’s 
teacher education learning system. We review the requirements in a sequential manner — from 
policies that influence the basic knowledge and skills that teachers obtain while still in high 
school to the content knowledge obtained in high schools, universities, and colleges, and finally 
to the institutions that impact the integrated teaching practices that teachers learn and apply 
during their teacher preparation program and induction experiences. 


The knowledge required to earn a single- or multiple-subject teaching credential is 
standardized. All credential holders must have a high school diploma (or equivalent), a 
bachelor’s degree, and they need to demonstrate proficient levels of knowledge in basic 
knowledge and skills, content knowledge, knowledge of reading development, and knowledge 
of English language development. The demonstration of these latter bodies of knowledge is 
usually accomplished through the passing of assessments or coursework, or both. 


Further, all of the state’s knowledge assessments have been created through well- 
established test development processes and the test blueprints, sample items, and various 
supporting validity documents are available online (CTC, 2007; California Educator Credentialing 
Examinations, n.d.; Le & Buddin, 2005; Pearson Education, 2014). Also, required teacher 
knowledge assessments are aligned to the CSTPs (Sandy, 2016). 


While there is strong standardization of requisite knowledge, there is also variation in 
the particular learning opportunities that pipeline institutions provide to aspiring teachers. We 
begin with the earliest pipeline institutions, K-12 secondary schools, followed by community 
colleges and 4-year IHEs. 
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K-12 secondary schools. Current course requirements for California high school 
graduates are listed in Appendix C. These requirements vary by subject and, given our review, 
vary by school and school district. In the case of subject matter variation, there are different 
knowledge requirements for different subjects. Although high school graduates may obtain 
relatively broad knowledge in the fields of English and social studies because they are required 
to successfully complete three years of coursework in these content areas, students’ mastery of 
subjects like math, science, the arts, and career/technical education may be narrower due to 
lower coursework requirements. Additional variation in aspiring teachers’ knowledge may 
occur because both K-12 school districts and IHEs may set higher graduation requirements and 
admissions standards, respectively, contributing to aspiring teachers’ mastery of the KSCs. For 
one, public and private 4-year colleges and universities in California often set admissions 
requirements that exceed California high school graduation requirements. For example, 
admission standards for the California State University (CSU) and University of California (UC) 
systems, which issued over 40 percent of California’s multiple- and single-subject preliminary 
and intern credentials in 2016-17, call for students to complete, with a grade of “C” or better, 
what are commonly known as the “A-G” subject requirements — a set of courses that are 
“academically challenging, involving substantial reading, writing, problems and laboratory work, 
and show serious attention to analytical thinking, factual content and developing students’ oral 
and listening skills” (The California State University, n.d.; University of California, n.d.). So, 
whereas California state statute requires K-12 students to complete two years of mathematics, 
including one year in algebra I, to obtain a high school diploma, admissions into the UC and CSU 
systems requires the completion of three years of mathematics courses, including algebra, 
intermediate algebra, and geometry. 


While completion of UC and CSU’s “A-G” subject requirements is not mandated by state 
law, many school districts across the state of California have enacted local graduation policies 
that require their students to meet the “A-G” subject requirements in order to receive a high 
school diploma from the district. For example, San Francisco Public Schools requires all students 
— regardless of whether they will pursue a postsecondary degree in the UC or CSU system — to 
complete the “A-G” subject requirements with a grade of “D” or better in order to earn a high 
school diploma (SFUSD, n.d.a). Yet, the level of mastery a student gains upon achieving a “D” in 
a class is questionable. For example, the SFUSD mathematics department suggests that math 
teachers use a 4-point holistic rubric to evaluate student work. The math department states 
that if a teacher wants to convert rubric scores to grades, a grade of “D” corresponds with 
“minimal understanding” — a “1” on the 4-point holistic rubric. Moreover, “many teachers 
require students revise their work when they receive a score of 0, 1, or 2 so that they can show 
progress toward mastery of the standards” (SFUSD, n.d.b). As such, it is likely that a student 
that obtains a grade of “D,” and even “C,” may have quite limited mastery of the content 
knowledge in a particular mathematics course. 


Local school districts also may set higher graduation requirements in other areas that 
impact students’ attainment of specific KSCs that are necessary for effective teaching. For 
example, students in Los Angeles Unified School District and seven other California school 
districts are required to complete an ethnic studies course to be eligible for high school 
graduation. Such a policy undoubtedly influences students’ KSCs, particularly those related to 
the CSTP domains of Engaging and Supporting All Students in Learning, Creating and 
Maintaining Effective Environments for Student Learning, and Planning Instruction and 
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Designing Learning Experiences for All Classrooms. Current publicly available data does not link 
aspiring, preservice, or practicing teachers to their K-12 secondary school, so we neither know 
whether certain high schools are disproportionately contributing to the state’s teacher 
workforce nor whether or how certain preparation pathways are shaped by the quality of 
secondary education in the state. 


Community colleges and 4-year IHEs. California state law requires aspiring K-12 
teachers to possess both a high school diploma and a bachelor’s degree. Outside of the 
accreditation process and previously discussed institutional policies around admissions 
requirements, institutional rules related to course and graduation requirements play a 
substantial role in the KSCs that aspiring teachers develop as they are pursuing a postsecondary 
degree. At a broad institutional level, community colleges and IHEs set course-hour 
requirements and grade point average (GPA) requirements in order to obtain, in the case of 
community colleges, an associate’s degree or to transfer into an IHE or, in the case of IHEs, to 
obtain a bachelor’s degree. 


31 | Getting Down to Facts II 


auljadig uolyeonpy Jayoea| s 


PIUJOJED JO MAA S, 


SWeIZOJg UOINPU] PUe BdIAIAaSaJdg Ul S41SD pue 


wayshs Vy | 


ce 


‘Vd L ‘Sdd 1 elusogied “y aunsiy 


~Aqyiqisuodsar ;euorssayoid Suneysuousg 179 


70} sonmpqisuodsax [euorsseyord SuseueW| 9°9 
“quapnys yioddns 0} sanrunumuos [Roo] Sursesuq $9 


Joyeonpe jeuorssayosd 
v se Surdopoaad :9 dLSO 
+ ATOUIT] aIVYS 0} UOTVULIOJUT JUOUTSSasse SUIS_) 1S 
“UL JSISSB 0} SOISOTOUYI} dTQuITeAR BUTS) 9°¢ 
SuIUIes] 1OF 


“""]ROS “JUOUISSOSSe-J[OS UI S}UOpNys [[B SUIATOAU] ¢°S 
*BUIUIVI] YSI[GRISO 0} eJep JWOUISSOSse SUISA-) $'S 
“YAN pue ATTENprATpUT YjOq “eIVp SUTMOIAOY €°¢ 

““WUOJ] BJBP JUSUISSasse SuizATeUe PUR SUNIATIOD 7S 


sjuopnys Sutssassy :$ dLSO 


Joyeonpe jeuorssayord 
® sv Surdopoaaq :9 Ad. 


““-gunjonys “yxa]U09 Moy ozATeUR ATTRILID 19 
***sa[O1 [BUOISsayord jowuS puke purysiopur) 9°9 
~--Aqyigisuodsag yeuorssayord ayeysuowaq ¢°9 
“**QATOAUT 0} USA puk MOY ayeISUOWOG $°9 
““S[k0s [eworsssyord ystiqeisd €°9 
“*Soseiq Pue sonfeA UMO OZIUBOIEY 7'9 
“gorjovid Surpoee} UO JOATJOY 1°9 


mK 


me 
~ 


“‘sasodind oy} Jo espoaymouy SsurAjddy [°¢ 


“pure sIoUIRI] YSI[suq JO spoou oy) Surssarppy 9°¢ 
*“SgIsoO[OUYo9} ‘sdomosal Surjdepe pue SuIsA S°¢ 
“o7e JeY} SasojeIs [PUOHONASUI SuIZTNA '€ 
““JUOPNYS oyey]IOB} O} UIN|NOLLM SUIZIURBIQ ¢°¢ 
“-quaurdopaaap JuUapNys JO ospoyMouy SulA[ddy 7'¢ 
“Joye yoolgns Jo spo; Mouy Sue.ysuowsgd [°¢ 


Sutures] Juopnys 
JO} 19}3euI JOofqns Surzrues1o0 
pue Surpueysiopun :¢ dLSO 


Sunsni{pe pue Suruses] JUSpNys SuLIOWUO 9° | 
~Ambut ysnospy Suppuryy [Royo SuNOWOIg S| 
““'So1so}Bs [BUOTJON.SUI JO AJOLILA B SUIS() fT] 
“year [nysuruvsur 0} Joyeul yoolqns SuNsOUUOD ¢° | 


SUIUIvO] 
UI syUSpNys [Je Sunoddns 
pue Sursesuq :| dLSO 


““-JoLid ,sjuapnys 0} Surusesy SuNIOUUOD Z| 


“UL UIOY} OSeSUd 0} SJUSPNys JO Spo] MOUY SuIS-) ["] 


em 


‘a 
‘a 


x x 


x x 
x 
x x x 
x 
xX xX 


ssossy 
Ajdd pI]peay 3 yovay 
Vv 


(Vd LI¥D Pastsody) JUITUSSISSY DIULULIOJ1Ig SUIYIVA TL, 


Suruses] 
quapnys Surssassy :$ Td 


Suruses] yuopnNys 
Joy Joye JOo[qns Suiziues.10 
pue Surpurjysiopuy :¢ad.L 


Surusregy 
UI syuSpNys [[e Sun1oddns 
pue Suisesuq :| db 


“-Byep 

jusUssasse 7O¢ Pur “LI ‘“dSAl ‘dal 98 8'¢ 
“pyep JUdUUssosse Tq Jordsoquy /°¢ 

“sysiperoads YIM YOM 9°¢ 

Ae A[OUIN Ul UOIPULIOJUT JUSUISSOSse 9S—) S°¢ 
---yzoddns 0} ASopouyse} asQ 7S 

“* JUOUISSASSB-J[9S UI SJUAPNYS DATOAU] €°¢ 
“Rep JUOUISSasse oZATRUR PUP 199]JOD ZS 
“*-sasodind ay} Jo asparmouy Addy ["¢ 


““SaIdayBS JO ISpo|MOUY deIJSUOLUIAG BE 
*Kovsoy] [eUsIp doyaasp pur [apoy 1 
***sgamnosal jdepe pur as—) 9°¢ 

““-qioddns 0} umynoruns ydepy ¢°¢ 

““"IOJ ayeIOGRI[OO pue INsUOD f'¢ 

“-yuouro[dun “ustsop “‘uR[g €°¢ 

““sjUOpNys jnoge dspoyMOUy 9s-) 7° 

“"-Jayeul joofgns Jo ospa,MouyY ayBsSUOMIAG ['¢ 


““qsn{pe pue Suruses] 1OWUO 8] 
“-sartunyioddo syuapnys apraoig L' | 
*qUSUTUOITAUS SuTUIee] 2AtJOddns apraolig 9°] 
*BULUTY) SANRIO PUR [BOI}LIO 9J}OWIOIg S| 
“sgisoqes [eUOTjON.YSUT oyeLIdo1dde asA ' | 
“""$]XOJUOD JI] [BAI 0} JOOUUOD €" | 

“S]UOpNys YJLM UOTBOTUNUTUOD UTeUIeYY Z| 
““S]UOpNys JO ospo|Mouy A[ddy | "T 


uoneiedaig 19499], 


While data related to the number of teaching credentials issued by each institution in a 
given school year is publicly available, data related to the undergraduate institutions that 
California teachers attend is currently not publicly available. As such, it is impossible to review 
the coursework requirements and graduation requirements of, say, the top five IHEs where 
novice teachers obtain their bachelor’s degree. It is also impossible to know the proportion of 
teachers who obtained a bachelor’s degree that started in a community college and transferred 
to an IHE. From publicly available data on the CTC data dashboard, we do know that, in 2016- 
17, the CalStateTEACH programz4 issued the largest number of multiple-subject credentials, 
followed by Fresno State, National University, Loyola Marymount University (LMU), and CSU 
Fullerton. We do not know what these candidates’ majors were so we cannot assess the degree 
to which there is variation or similarity in learning opportunities provided to aspiring teachers. 
Given this lack of data, we explored coursework and graduation requirements for students 
pursuing a liberal studies/interdisciplinary major at two institutions that issue a large number of 
multiple-subject teaching credentials each year: Fresno State and LMU. Table 2 summarizes the 
undergraduate coursework and graduation requirements for candidates at the two 
aforementioned institutions. We present this information with the understanding that these 
institutions that award large numbers of teaching credentials may or may not be the 
institutions with a large number of undergraduates who complete their bachelor’s degree and 
then pursue a teaching credential. 


At Fresno State and LMU there are many similar required learning opportunities as well 
as differences. Both IHEs require general education coursework that is divided into various 
types of knowledge — foundations, explorations, breadth, integrated, etc. Both institutions 
emphasize opportunities to learn to write as well as learn history, mathematics, science, and 
literature in the blended parts of the liberal studies/interdisciplinary major. There is, however, 
variation as well. LMU requires 25 more units than does Fresno State in order to graduate. 
Fresno State requires courses in physical education, linguistics, secondary language learning, 
and students with special needs. LMU does not require these courses as a part of the major; 
however, LMU includes psychology, geography, world history, as well as American and 
California history as well as more required hours in science and the visual arts, music, dance, 
and movement arts for children. While these two institutions are a small number of the many 
that educate aspiring teachers, it is clear that there is significant variation in the learning 
opportunities undergraduates might experience on their path to becoming a teacher. 


14 See https://www.calstateteach.net/ for information about this online teacher education program. 
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Planned Variation and Professional Choice in Integrated Teaching Practices 


Knowing the specific subject matter one will teach is necessary, but not sufficient for 
being able to teach it (Ball et al., 2008). One must also be able to make that knowledge useable 
in the classroom in the teaching practices delineated in the CSTPs. California’s current policies 
implicitly acknowledge this; using member-only Certification Data Maps from the National 
Association of State Directors of Teacher Education and Certification, we found that California is 
one of 27 states that require novices to meet a state requirement concerning performance 
assessment (NASDTEC, 2018). It is unclear how many of these states require a performance 
assessment like California’s TPA; however, a recent scholarly article suggests only seven states 
require candidates pass a TPA for certification (Goldhaber et al., 2017). Our document analysis 
suggests that while knowledge is highly standardized in the teacher education pipeline, there is 
significant variation in the specific practices for which a candidate must demonstrate 
competency prior to her becoming the teacher of record. It would be difficult to say, for 
example, whether every 8th grade science teacher can plan engaging lessons equally well in the 
subtopics of geology, force and motion, and energy, or whether all elementary teachers are 
able to carry out a whole class discussion on the theme of loss, or whether teachers of all 
grades are able to communicate the academic progress of a student with a learning disability to 
that student’s parent. So, while the CSTPs describe the general KSCs beginning teachers need to 
know and be able to do on their first day, the learning system allows for a good deal of 
variation around the exact KSCs any one teacher will be able to competently carry out on her 
first day. 


That variation stems, in part, from sanctioned professional choices embedded 
throughout California’s teacher education pipeline. These choices allow for variation in the 
specific teaching practices, topics, and in some cases, subject areas in which candidates 
demonstrate their KSCs. Systematic professional choice exists at three locations in the teacher 
education system. There is choice about how to demonstrate competency in the specific 
coursework and field-based experiences candidates have during the TPPs. There is choice 
within and across the performance assessments that are a part of preparation programs. And, 
when a candidate continues her early career learning during her induction program, there is 
professional choice about exactly which TPEs she will focus on in her ILP. 


Programs align their learning opportunities to the TPEs during the accreditation process. 
However, there is significant flexibility in how the TPEs are met. As just one example, LMU, a 
large private TPP, requires prospective elementary teachers to take a three-credit course prior 
to entry into the program focused on cultural paradigms in education, including culturally 
relevant pedagogy. In contrast, CSU Fullerton does not have a stand-alone course on 
multicultural education and instead integrates this course content into more than one course. 
Research suggests that structural differences may matter to preservice teacher learning (e.g., 
Heafner & Petty, 2016); however, we are unaware of empirical evidence to support one 
structure versus another. Despite this gap in the literature, it seems possible that different 
structures may lead candidates to develop different levels of KSCs around teaching diverse 
learners. 
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Second, while TPPs are all accredited on the basis of California’s TPEs, which are aligned 
with the CSTPs, and all preservice teachers must pass a TPA (CTC, 2017b), there are three 
different TPAs used in California at this time (Campbell et al., 2016).15 The actual tasks 
candidates carry out as a part of their TPA must address all six of the TPE domains, but tasks 
vary and programs and candidates decide exactly how they will carry out the tasks within a 
given TPA model (Campbell et al., 2016). For example, for edTPA, used by approximately 3,500 
candidates in CA during 2015-16 (CTC, 2017c), candidates are allowed to make different 
decisions about the specific teaching practices and topics submitted for evaluation across the 
four tasks, each of which is carried out with a focal class of students. The edTPA’s second task 
for multiple-subject candidates focuses on instructing and engaging students in literacy 
learning. Preservice teachers are allowed to select the essential literacy practice they wish to 
focus on as well as whether the practice occurs in a writing or reading lesson. Candidates can 
choose strategies as diverse as using a rubric to revise a written essay or identifying a 
character’s personality traits based on what they say and do. In contrast, the CalTPA, a TPA 
used by 9,000 candidates in CA during 2015-16, has three tasks that require work with K-12 
students — designing instruction, assessing learning, and a culminating teaching task. Each task 
requires novices to show a variety of TPEs. For multiple subject candidates, they must carry out 
these tasks in ELA and math, but they can choose which subject they will do for each task, and 
there is no requirement to focus on a specific essential literacy practice. And just as with edTPA, 
CalTPA candidates are allowed to show their competency in either a reading or a writing lesson. 
This variation means that a candidate assessed on CalTPA might show evidence of competency 
teaching students to distinguish common forms of literature while another candidate shows 
evidence she can support students learning to tell time. These are different practices and likely 
require the use of different pedagogical strategies. It is important for all candidates to be able 
to competently teach both of these practices, but the current TPA allows the candidate to 
select which one they choose. 


1s There were at one time four TPAs: The California Teaching Performance Assessment (CalTPA; CTC, 2018d) which 
is currently undergoing revision and field testing; the Fresno Assessment of Student Teachers (FAST; Torgerson, 
Macy, Beare, & Tanner, 2009); the Performance Assessment for California Teachers (PACT, 2018); and the edTPA 
(2018). PACT was the precursor assessment to the edTPA, however, PACT is no longer supported by its developers. 
Stanford, the developers of PACT, own edTPA (Campbell et al., 2016). 
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Table 3. Tasks, Topics, and Examples of Allowable Teaching Practices in Two TPAs 


edTPA CalTPAi6 
1. Planning for instruction and 1. Subject-specific pedagogy 
assessment 2. Designing instruction 
Tasks 2. Instructing and engaging 3. Assessing learning 
students in literacy learning 4. Culminating teaching task 
3. Assessing student learning 
Topic Reading or writing Reading or writing or math 
Examples of Revise a written essay, or determine Tell or write time, create a 
allowable what characters are like by paragraph and include simple 
teaching comprehending what they say anddo details, distinguish common forms 
practices of literature 


The opportunity to make choices about the specific teaching practices to focus on and 
demonstrate competency in continues during the induction phase of the pipeline. In order to 
be granted a clear credential, beginners must continue to learn and develop their KSCs through 
progress on a personalized ILP. ILPs are developed at the end of the preservice program and 
serve as the starting point for the mentor and beginning teacher to identify areas of strength 
and weakness, set measurable goals, and work together to improve the beginner’s teaching 
practice (CTC, 2017a). Over the induction program, teachers must work on each of the six TPE 
domains; however, the specific practices within these domains are allowed to vary. Here again, 
there is a focus on integrated teaching practices; however, there is little standardization 
around the specific KSCs a new teacher will be able to carry out at the end of the induction 
phase. 


To understand the degree to which this variation might matter for novices’ skills, we 
looked for research articles that link performance on the three California TPAs to beginning 
teaching practice. We were unable to find any such articles in peer-reviewed journals, however, 
we found one article that documents a relationship between preservice teacher performance 
and student achievement on one of the TPA assessments. On the edTPA in Washington state, a 
mixed, but positive relationship was found between teachers’ edTPA and value-added (VAM) 
scores taken from the classrooms in which they became the teacher of record (Goldhaber et al., 
2017). The report found that passing the edTPA was predictive of a teacher’s later VAM score in 
math when using continuous scores from the edTPA. When using the pass/fail distinction 
Washington uses for certification, edTPA scores were predictive of VAM in reading, but not in 
math. Scores in both subjects predicted whether a candidate was employed in the next year. 
These mixed findings that are sensitive to subject matter differences and statistical modelling 
and measurement assumptions typify the complexities of understanding the relationship 
between teaching practice and other related constructs (c.f., Rowan & Raudenbush, 2017). If 


16 The CalTPA is under revision. During 2017-18 it was undergoing a field test and will be implemented in revised 
form in 2018-19. This information is based on the current version. 
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the findings in Washington generalize to California, it is possible that the choices allowed within 
California’s TPA assessments produce performances that are related to the strength of a 
beginner’s teaching practice. While we were unable to find evidence in California on the current 
edTPA, its precursor, PACT, does have two small scale studies linking PACT scores to value- 
added estimates of the first years of teaching (Darling-Hammond, Newton, & Wei, 2013; 
Newton, 2010). However, there is very little research that would establish that relationship and 
no research that has specifically investigated how choices within TPAs relate to actual 
differences in quality teaching or their eventual students’ learning. Additional research will 
need to investigate these potential relationships to better understand the importance of this 
allowable variation. At this point we know that the allowable variation means that teachers can 
pass their TPAs by demonstrating significantly different practices and that many teaching 
practices are not assessed by the TPA. 


Looking systematically at policies that guide implementation of the CSTPs, it is clear 
that, at a high level, there is standardization in what teachers are required to know and be able 
to do. There is also allowable variation based on professional discretion and individual learning 
needs at the more detailed level of specific practices and topics in the K-12 curriculum. 


There is one final source of variation in the KSCs beginners develop: the pathway they 
follow. As candidates make their way through traditional and intern pathways, they engage 
different pipeline institutions. The pipeline institutions’ ability to provide candidates 
appropriate learning opportunities add additional variation to on-the-ground implementation 
of the CSTPs. 


Similarity and Variation in Traditional and Intern Pathways 


While 72 percent of California’s 2016-17 single- and multiple-subject credentialed 
teachers completed a California-based traditional pathway described previously, the intern 
pathway was completed by 16 percent of newly credentialed teachers in 2016-17. Like 
traditional programs, intern programs are accredited by the CTC. There are two types of intern 
programs: district and university. As of 2016-17, there were seven local education agencies:17 
with CTC-accredited district intern programs for single-subject and multiple-subject teaching 
credentials, from which just 284 credentials were issued (129 single subject and 155 multiple 
subject). Tulare County Office of Education credentialed the largest proportion (27 percent) of 
district intern-credentialed teachers in 2016-17. In contrast to district intern programs, there 
are many more CTC-approved institutions that offer a university intern program: 21 CSU 
institutions,is three UC institutions, and 25 private/independent institutions. In 2016-17, 
university intern programs produced 2,281 credentialed teachers. 


17 Bay Area School of Enterprise (Reach Institute), High Tech High, Los Angeles County Office of Education, Los 
Angeles Unified School District, Mt. Diablo Unified School District, Sacramento County Office of Education, San 
Joaquin County Office of Education, and Tulare County Office of Education. 

is This figure does not include the CalTEACH program. 
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Intern programs are a path to a teaching credential that allows an individual to 
complete her teacher preparation coursework concurrent with her first year or two in a paid 
teaching position and after completion of 120 hours of preservice coursework (CTC, 2008; 
2013). All of the assessment requirements for traditional pathways (e.g., CBEST or similar, CSET, 
RICA, TPA, etc.) apply to the intern pathway. However, the learning opportunities in specific 
education courses, student teaching experiences, and other mentoring and practice teaching 
experiences are different for an intern candidate. Perhaps most importantly, the education of 
the intern largely takes place in the daily context of the intern’s own classroom and school 
building. Thus the K-12 school largely provides the candidates with their opportunities to learn 
from practical placements. Further, additional coursework is frequently carried out while the 
individual is a teacher of record. This means the candidate has a full-time teaching job and is a 
novice, circumstances that we know anecdotally can be challenging. 


Intern programs conceptualize and monitor candidates’ learning opportunities in 
different ways than do traditional programs. In traditional pathways, there are courses that 
specify what the preservice teacher should learn, providing specific learning opportunities 
organized around topics. In the intern pathway, school districts that employ interns in CTC- 
accredited intern programs are required to develop and implement a Professional Development 
Plan for interns in consultation with a CTC-accredited TPP that includes an annual evaluation, 
plans to complete necessary courses, additional instruction in child development and teaching 
methods, special education programs for pupils with mild or moderate disabilities (for K-6 
interns only), and additional instruction in the culture and methods of teaching bilingual 
children (bilingual interns only). There are traditional courses in interns’ professional 
development plans, but courses can be augmented with other learning structures as well. 


In addition to these broad requirements, district intern programs must provide 144 
hours of support and supervision per school year including coaching, modeling, and 
demonstrating within the classroom, assistance with course planning and problem solving 
regarding students, curriculum, and development of effective teaching methods. In doing so, 
the mentor must provide a minimum of two hours of support/mentoring and supervision every 
five instructional days (Intern Teaching Credentials, 2014). 


While beginners pursuing a teaching credential through the intern program must 
complete these requirements, it is important to recognize that interns are in classrooms and 
teaching students prior to the mastery and completion of the milestones that contribute 
toward the accumulation of the KSCs necessary to be an effective teacher. Interns do, however, 
have many more hours interacting with students prior to the candidate being certified than do 
candidates pursuing traditional pathways. Unfortunately, we are unable to examine the efficacy 
of either pathway due to data constraints; a point we return to later. 


To summarize, California has a teacher education pipeline characterized by a 
professional, developmental view of teaching that is embodied in the CS7Ps. This view of 
teaching is implemented through various policies including assessment and accreditation 
requirements that generally produce high-level system alignment with the CS7Ps. The 
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knowledge teachers are required to have is also tightly regulated and available for public 
inspection through various testing documents and state reports. There is, however, a good deal 
of variation in the specific knowledge candidates develop that falls outside the state’s teacher 
tests. All across the pipeline, from high school to IHEs to TPPs, there are differences in how 
many courses, which courses, what institution is providing the beginner learning opportunities, 
and what levels of achievement a candidate is required to reach. Further, there is significant 
allowable variation in the specific integrated teaching practices beginners are required to 
demonstrate. Such ground-level variation begs the question: does this variation influence how 
well beginners are prepared? We now turn to the relatively small amount of empirical data we 
can bring to bear on this question. 


Judging the Preparedness of Beginning Teachers 


One goal of any teacher education system is to prepare teachers who are ready to teach 
all children without doing educational harm on their first day as the teacher of record. There is 
disagreement in the field as to the most reasonable ways to judge this. Educational reformers, 
national accreditation regulations and state officials often argue that the preparation of 
teachers should be judged based in part, on the beginner’s impact on student achievement. 
Any such metric can then be aggregated up to the program level to identify areas of strength 
and weakness in programs (CAEP, 2016; Deans for Impact, 2016). Others reject using student 
achievement data for individual teachers or for programs as methodologically fraught and 
therefore ill-advised, due in part, to the strong sorting of prospective teachers to TPPs as well as 
sorting that occurs after the candidates leave their programs — sorting that occurs at the 
district, school, and student level (Koedel, Parsons, Podgursky, & Ehlert, 2012; Goldhalber, 
Liddle, & Theobald, 2013; Mihaly et al., 2013). Additional methods of judging the preparedness 
of beginners include using the classroom observation scores of beginning teachers, survey data 
from the beginners and their supervisors (Allen, Coble, & Crowe, 2014), and assessments of 
candidates’ knowledge (Gitomer & Qi, 2010). 


Regardless of one’s views of linking beginners to their K-12 students’ academic success, 
California does not have a statewide database that links TPP completer records with their 
students’ performance data, nor is there systematic classroom observation data with common 
instruments collected in the state. Thus, to gain insight into the preparedness of beginning 
teachers in the state, we first review publicly available data and published reports on 
candidates’ performance on knowledge tests. These data suggest that no matter what path 
beginners take through the teacher education system, if they complete that path, they are 
similarly prepared. It is important to note that by requirement, completing a pathway means 
the novice has passed the mandated knowledge and practice-based assessments (see p. 30 for 
a recent report [CTC, 2017i]). 


We subsequently draw on two different bodies of stakeholder survey evidence on how 
well beginners are prepared. These survey data, like the publicly available data and published 
reports on candidates’ performance on knowledge tests, similarly suggest similar levels of 
preparation across specific CSTPs. That is, the variation in institutional and assessment 
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requirements documented in the previous section does not appear to produce much variation 
in beginners’ perceptions of their preparedness. Program completer survey data suggests 
novices have similarly positive perceptions of their preparation across all teaching 

practices. New survey data collected by the Getting Down to Facts II project suggests that 
principals do not agree with the level of beginner preparedness reported by beginners 
themselves. In particular, principals perceive beginner preparedness as systematically lower 
than do beginners and principals perceive greater variation in beginner teachers’ preparedness 
across teaching practices. However, beginning teachers and principals agree on the specific 
CSTPs for which beginner teachers are most and least prepared. 


Knowledge 


Based on publicly available data on the CTC’s Title Il dashboard, preservice teachers 
generally pass the knowledge assessments required by the state for both multiple- and single- 
subject candidates. Candidates preparing for elementary teaching are required to pass RICA, 
CBEST, and three CSET exams. In Table 4, we again profile the two TPPs that produce a large 
number of teacher candidates across their intern and traditional pathways — Fresno State and 
Loyola Marymount University. These data show that across the four pathways, between 85-100 
percent of candidates pass all of their required assessments. CBEST and CSET pass rates are all 
100 percent, with a small proportion of candidates failing the RICA assessment. All of the 
average program scaled scores are well above the state’s mandated cut scores, although there 
is some variation within and across programs. 


For example, although pass rates across assessments are relatively similar between 
institutions within pathways, the average scale scores of LMU students across all assessments 
are higher than those of students at Fresno State, across both the traditional and intern 
pathways. Within institutions, Fresno State students in the traditional pathway have higher 
CSET but lower CBEST and RICA scores than those in the intern pathway. In contrast, LMU 
students in the intern pathway have higher average scale scores than those in the traditional 
pathway. Despite this variation, what is striking in Table 4 is the high pass rates across 
assessments, institutions, and pathways. Further, all institutions and pathways’ average scale 
scores are substantially higher than the state-mandated cut scores. 
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Table 4. CBEST, RICA, and CSET Scores and Passing Rates for Traditional and Intern Pathway 
Multiple-Subject Teacher Credential Students in Two Large IHEs, 2015-16 
Multiple-Subject Teacher Credential Students 
Takers Passers Pass Low High Cut Average 
Rate Score Score Score Scale 


Score 

Fresno State - 

Traditional 
CBEST 264 264 100% 60 240 123 147 
RICA 126 107 85% 100 300 220 230 
CSET Multiple 133 133 100% 100 300 220 236 
Subjects Subtest | 
CSET Multiple 133 133 100% 100 300 220 245 
Subjects Subtest II 
CSET Multiple 133 133 100% 100 300 220 238 
Subjects Subtest III 

Fresno State - Intern 
CBEST 77 77 100% 60 240 123 149 
RICA 26 25 96% 100 300 220 233 
CSET Multiple 30 30 100% 100 300 220 231 
Subjects Subtest | 
CSET Multiple 30 30 100% 100 300 220 238 
Subjects Subtest II 
CSET Multiple 30 30 100% 100 300 220 234 
Subjects Subtest III 

LMU - Traditional 
CBEST 155 155 100% 60 240 123 160 
RICA 87 76 87% 100 300 220 233 
CSET Multiple 91 90 99% 100 300 220 242 
Subjects Subtest | 
CSET Multiple 91 90 99% 100 300 220 249 
Subjects Subtest II 
CSET Multiple 90 90 100% 100 300 220 242 
Subjects Subtest III 

LMU - Intern 
CBEST 268 268 100% 60 240 123 169 
RICA 166 156 94% 100 300 220 236 
CSET Multiple 154 154 100% 100 300 220 244 
Subjects Subtest | 
CSET Multiple 154 154 100% 100 300 220 252 
Subjects Subtest II 
CSET Multiple 154 154 100% 100 300 220 243 


Subjects Subtest III 


The high pass rates and similarities within programs in the publicly available Title Il data 
mask variation in candidates’ actual knowledge due to how scores are reported as well as state 
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policy. First, state policy requires a beginning teacher to pass all the required exams (or have 
met alternative thresholds). This means that by definition, all credentialed teachers will 
eventually pass these tests and higher scores are not rewarded in any way — passing by 1 point 
or passing by 20 points are the same, from a policy perspective. The fact that all candidates 
pass their required assessments is not particularly helpful information for improving the 
pipeline. What is more helpful is knowing how many times candidates take these assessments, 
the variation in candidate’s knowledge within and across programs and pathways, as well as the 
particular tests and areas of the tests with which they struggle. If, for example, beginners 
preparing to teach elementary school-aged students in California have the same difficulties 
with mathematics that teachers in a nationally representative sample do (Hill, 2010), such 
difficulties should inform program faculty and policymakers as they attempt to support 
preservice teacher mathematical knowledge. If some pathways’ candidates take assessments 
an average of 1-2 times, and other pathways’ candidates take assessments an average of 2-3 
times, this has implications for how the institutions in those respective pathways need to 
support candidates with additional learning opportunities or other types of program-level 
policy changes. The Title Il data report very limited variation in candidates’ knowledge at the 
program or pathway level and provide no information on the number of times candidates take 
the required assessments before passing. 


A second reason there is limited information in the Title II data is because programs 
have generally moved to policies that require candidates to have passed specific assessments 
before the TPP reports on those students for Title Il. For instance, many programs require 
candidates to have passed CBEST before the candidate begins the TPP. This results in a 100 
percent passing rate on CBEST. This masks variation that might exist within and across 
programs and pathways. A recent report shows that there can be stark differences between 
first time and cumulative passing rates at the state level. On the multiple-subject CSET exam, 
almost 20 percent more candidates eventually passed than passed the first time they took the 
exam (CTC, 2016a, p. 21). This report suggests there is variation in test scores for at least some 
candidates and that there are differences in passing rates for candidates of different racial and 
ethnic backgrounds, with African American candidates generally having the lowest first-time 
and cumulative passing rates. Despite these important state-level data that show variation 
beneath the Title Il high pass rates, the report is difficult to use for program and system 
improvement because it does little to trace trajectories of candidates through programs or 
system pathways. 


Publicly available Title II data also show some differences in scaled scores across these 
same knowledge assessments. The state publishes data associated with a number of content 
knowledge assessments — e.g., CBEST, CSET, and RICA scores. It is possible to disaggregate these 
data by program type (TPP, district intern, university intern) as well as program sponsor (e.g., a 
university of other educational organization accredited to administer a teacher preparation 
program). These data, shown in Table 5, suggest that there is some variation in mean scaled 
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score by program sponsor and type. The UC and “other sponsors”19 programs tend to have the 
highest mean scores. And in traditional programs in California, which produced 72 percent of 
preliminary single- and multiple-subject teaching credentials in California in 2016-17, the 
highest scores are in the UC system, followed by independent schools and then the CSU system. 


This pattern does not hold across these institutions in the “alternative pathway.” For 
example, on the CSET I, the two highest mean scores were in “other program sponsors” 
programs and CSU programs. In contrast, the highest two mean scores on the CBEST were in 
“other program sponsors” programs and the UC programs. It is unclear what to make of these 
patterns. They could be the result of specific curricular interventions programs have 
implemented to support preservice teacher learning. They could also be the result of 
candidates being sorting into TPPs such that candidates with stronger basic knowledge and 
subject-specific knowledge wind up in specific TPPs or pathways. Alternatively, specific K-12 
and higher education pipeline institutions may be educating future teachers in more or less 
effective ways, and this is visible in the knowledge scores of preservice teachers. The publicly 
available data does not allow us to sort between these hypotheses, a point to which we return 
later.20 


The data also suggest that on average, candidates who have not yet completed their 
clinical coursework (i.e., candidates in earlier stages of their program) have lower scores on the 
required RICA, CBEST, and CSET assessments than the scores of other candidates who were 
further along in their programs. This may imply that mean scores rise to passing levels by the 
time the candidate has completed her TPP. Depending on whether preservice teachers are 
studying for the assessment or simply learning the relevant content assessed by the test, we 
may view this shift in scores as either short-term test preparation or valuable learning. 


Finally, publicly available reports provide CBEST, CSET, and RICA first-time and 
cumulative passing rates by demographic variables of test takers (CTC, 2016a). In particular, the 
reports provide participation data as well as first-time and cumulative pass rates across all 
teachers pursuing a credential between 2010 and 2015 based on the following categories: 
highest level of high school math completed, number of high school literature courses, number 
of high school writing courses, number of high school oral language courses, high school GPA, 
high school attendance, college math-taking, college GPA, number of college literature courses, 
number of college writing courses, number of college oral language courses, education level, 
years away from college, professional preparation, test preparation course taking, employment 
status, reason for taking the exam, type of credential sought, father and mother’s education, 
best language, gender, and ethnicity. 


19 “Other sponsors” are local education authorities such as the Bay Area School of Enterprise (Reach institute), High 
Tech High, districts (e.g., San Diego Unified School District and Mt. Diablo School District), and county offices of 
education (e.g., Santa Clara and Stanislaus County Offices of Education). 

20 The state’s teacher knowledge assessments are criterion-referenced assessments and focus on reliability at the 
cut score that determines passing or failing. Scores far from the test’s cut score may be less reliable than those 
around the cut score and should be interpreted cautiously. 
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These data provide valuable information on the demographic characteristics of students 
who take and pass the basic skills, content knowledge, and knowledge of reading development 
assessments. However, it is impossible to ascertain the extent to which different types of 
undergraduate institutions or TPPs better prepare different types of students. For example, do 
students with a high school GPA of less than 3.0 cluster into certain TPPs? Do some TPPs doa 
better job of preparing students with a high school GPA of less than 3.0? Currently, the publicly 
available data do not allow for an understanding of the correlations between assessment pass 
rates for students of different demographic backgrounds pursuing different routes within the 
teacher preparation pipeline. Additionally, the publicly available data are pooled across years 
(2010 to 2015). As such, it is not possible to explore variation in assessment success across 
years. In short, we can neither see if California is getting better or worse at helping to prepare 
certain demographics of teachers, nor whether certain types of programs or pathways are 
getting better or worse at preparing certain demographic groups of teachers. 
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Perceptions of Preparedness 


Another source of information regarding the preparedness of beginners comes from 
stakeholder’s views of preparedness. Publicly available data suggest beginners generally view 
their programs and pathways positively. Data gathered for the Getting Down to Facts II project 
suggest administrators tend to have less positive views of beginners’ preparedness. Despite 
these differences, there is some agreement between novices and administrators about the 
CSTPs teachers are most and least prepared to carry out. 


Beginning teacher perceptions of preparedness. Currently, the only data publicly 
available from which we can ascertain beginning teachers’ perceptions of preparedness to carry 
out CSTPs comes from the CTC multiple and single subject credential program completer 
surveys.21 These statewide reports include responses from beginning teachers who were 
recommended for a credential during the academic school year (September 1 to August 31). 
According to the CTC reports, 97 percent of those who were recommended for a credential in 
2017 responded to the 2017 survey. Every institution that sponsored a single or multiple 
subject program and had ten or more completers, or five completers with a 100 percent 
response rate during the survey administration time, was sent a program-level report of their 
own completers who responded to the survey. These program-level reports are not publicly 
available. Moreover, the reports do not provide any way to investigate how teachers’ 
perceptions vary by TPP, the teacher’s gender or race/ethnicity, or other relevant 
characteristics (e.g., undergraduate major, certification pathway, etc.). As shown in Table 6, 
newly credentialed teachers that responded to the 2017 CTC completer survey, on average, felt 
that their TPP prepared them “well.” Average perceptions across both multiple and single 
subject teachers aggregated to the domains of the CSTPs ranged from 4.0 to 4.3 on a 5-point 
Likert scale in which a 5 means they felt that they were “very well prepared.” We find little 
variation in preparedness across the various elements of CSTPs. When averaging responses 
from both multiple and single subject teachers, the elements that newly credentialed teachers 
felt their TPPs prepared them best for are associated with Standard 2: Creating and Maintain 
Effective Environments for Student Learning. The elements they feel least prepared for are 
associated with developing as a professional educator. Among specific CSTP elements, we find 
that beginning teachers felt most prepared to create a productive learning environment with 
high expectations for all students and to use effective instructional strategies to teach subject 
matter and skills; and they felt least prepared to identify and address special learning needs 
with appropriate teaching strategies and plan for instruction by incorporating all relevant 
IFSP/IEP behavior and academic information. 


We do find some variation between how well multiple and single subject credentialed 
teachers felt their TPP prepared them in relation to the CSTPs. Across all six CSTPs, single 
subject teachers, on average, had lower ratings of preparedness than multiple subject teachers. 
Differences between single and multiple credentialed teachers were largest in the following 
CSTP elements: meeting the needs of English learners; using knowledge of students’ strengths 


21 Available at https://www.ctc.ca.gov/educator-prep/completer-surveys 
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and prior experiences to engage them in learning; planning instruction based on students’ prior 
knowledge, academic readiness, language proficiency, cultural background, and individual 
development; and working with families to better understand and to support their learning. For 
each of these elements, single subject teachers had lower perceptions of preparedness than 
multiple subject teachers. It is also important to note that, across all CSTP elements, the 
variation in perceived preparedness among single subject teachers is greater than the variation 
in perceived preparedness among multiple subject teachers. 
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Principals’ Perceptions of Preparedness. In addition to the CTC Completer Surveys, the 
Getting Down to Facts Il project gained access to data from a survey of RAND’s American School 
Leader Panel (ASLP) in October 2017. While not publicly available, these data provide additional 
insight into principals’ perceptions of teachers’ preparedness to perform the CSTPs. The RAND 
ASLP are nationally representative panels of practicing K-12 public school principals who have 
agreed to participate in online surveys several times each school year. This survey 
administration had a response rate of 31 percent. However, sampling weights were generated 
for the ASLP survey to (a) help to adjust for nonresponse at both the panel recruitment and 
survey stages and (b) adjust for oversampling. Additional information about the principal 
sample, including information about non-response bias, is available in the Appendix D. 
Descriptive statistics from the ASLP data reveal that principals’ perceptions of teachers’ 
preparedness to meet the CSTPs are substantially lower than the perceptions of those teachers 
completing the CTC completer survey in the spring of 2017 (see far-right columns of Table 6). 
The elements that had the greatest teacher-principal gap in perceptions of preparedness were 
not isolated to a single CSTP, but rather cut across elements associated with all six CSTPs. 
However, as shown in Table 6, the highest and lowest rated elements for which beginning 
teachers and principals felt most and least prepared were relatively consistent across the RAND 
ASLP and CTC survey. 


Similar to the CTC completer surveys, we find that principals’ perceptions of teacher 
preparedness among the CSTP elements are lowest for teachers’ preparedness to identifying 
and addressing special learning needs with appropriate teaching strategies and to plan for 
instruction by incorporating all relevant IFSP/IEP behavior and academic information. This 
agreement is particularly important given that California TPPs were required only this year to 
integrate special education standards into the general education curriculum. Survey responses 
suggest that teachers and principals felt the preparation of generalists needed additional 
support. The new policies that integrate special education KSCs into general education courses 
can now be evaluated against the previous trend documented here. Overall, these results 
suggest that teachers and principals may be in general agreement about the areas in which 
teachers are the least and most prepared; however, there is wide variation in their agreement 
about preparedness in many other teaching practices. 


We also see a slight difference in the absolute agreement levels among RAND ASLP 
survey respondents and beginning teachers responding to the CTC completer survey. In 
particular, beginning teachers’ highest rated elements were associated with being “very wel 
prepared and the principals’ highest rated elements were associated with being on the low end 
of “adequately” prepared. In addition to the average ratings of preparedness, we also looked at 
differences in the variation among beginning teachers’ and principals’ perceptions of 
preparedness. We found slightly more variation among beginning teachers’ than among 
principals’ perceptions of preparedness: across all survey questions, the average standard 
deviation was 0.88 units for beginning teachers and 0.82 for all principals. 


|? 
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To gain further insight the gaps between principals’ and beginning teachers’ perceptions 
of preparedness, we also investigated the extent to which differences in principals’ and 
beginning teachers’ perceptions of preparedness were associated with differences in the 
schooling environments in which the teachers and principals were located. We were able to link 
the schools of the principals with school-level student demographic data derived from the CDE 
and compiled by members of Stanford’s GDTF II, Dr. Lucrecia Santibafiez and Erika Nakamatsu 
Byun. Specifically, we linked the RAND data with CDE data and then categorized the principals 
into quartiles based on four school-level student demographic characteristics from the CDE 
data: percent EL students, percent White student, percent of students proficient or above on 
state standardized ELA assessment, and percent of students proficient or above on state 
standardized mathematics assessment. We then looked descriptively at principals’ perceptions 
of beginning teachers’ preparedness to perform the CS7Ps across the quartiles of school-level 
student demographics. The figures generally show there is a slight positive trend between 
principals’ perceptions of preparedness and lower percentages of EL students, higher 
percentages of white students, and higher percentages of proficient students on the state’s ELA 
tests. 


Looking for carefully, we find one notable exception to this slight trend. Figure 5 shows 
that principals in the second quartile on percent EL students in the school consistently had 
higher perceptions of teacher preparedness across nearly all CSTPs. Given this finding and the 
relatively small sample size of respondents in the ASLP survey, we looked specifically at all 
respondents from the second quartile on percent EL students in the school to see if the higher 
ratings were driven by a small number of principals. We found that principals in two to four 
schools within the second quartile of the percent EL students in the school drove up the 
average for the second quartile group. Additionally, we found that the standard deviation for 
principal respondents in the quartile 2 of the percent EL students group approached 1.0 on 
nearly every CSTP, which was substantially larger than the standard deviations for the other 
quartiles (standard deviations, on average, between 0.55 and 0.80). 


Figure 6 shows that principals in the third and fourth quartiles of percent of White 
students in the school consistently had higher perceptions of teacher preparedness across 
nearly all CSTPs. This means that principals in schools that had higher proportions of White 
students had higher perceptions of teacher preparedness. A similar trend is found for principals 
in the third and fourth quartile on percent of students proficient or above on the state 
standardized ELA exam (Figure 7). However, we do not see similar trends among the different 
quartiles of percent of students proficient or above on the state standardized mathematics 
exam (Figure 8). 


These findings may suggest that school-level student demographics may mediate 
principals’ perceptions of beginning teacher preparedness. However, due to the small sample 
size and, thus, lack of analytical power, we are not able to more deeply engage in analyses that 
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would better inform the magnitude and significance of this relationship. Nonetheless, if 
additional data were collected and made available, it would be possible to better understand 
whether both principals’ and teachers’ perceptions of teacher preparedness are associated 
with the schooling contexts in which they come to work. 
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@ 2nd Quartile 3rd Quartile 4th Quartile 


Figure 5. Principals’ Average Ratings of Teacher Preparedness in CSTPs, by Percent of EL Students in School Quartile 
@ 1st Quartile 
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Figure 6. Principals’ Average Ratings of Teacher Preparedness in CSTPs, by Percent of White Students in School Quartile 


@ 2nd Quartile 3rd Quartile 4th Quartile 
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Figure 7. Principals’ Average Ratings of Teacher Preparedness in CSTPs, by Percent of Students Proficient or Higher in ELA in School 


Quartile 
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Figure 8. Principals’ Average Ratings of Teacher Preparedness in CSTPs, by Percent of Students Proficient or Higher in Math in School 


Quartile 
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Data to Inform Pipeline Improvement Efforts 


In recent years, much emphasis has been placed on the need to have better data to 
inform the improvement of teacher education nationally (Deans for Impact, 2016; Feuer et al., 
2013; Southern Regional Education Board Teacher Preparation Commission, 2017). These 
reports, and others like them, suggest that disaggregated data related to a state’s valued 
educational outcomes is critical to improving teacher preparation, both within TPPs and across 
the institutions that participate in the teacher education pipeline. 


California collects and makes available a great deal of information that might be used to 
judge the quality of the teacher education system against the five criteria proposed previously. 
A full analysis of all the state’s data surrounding the teacher education pipeline is beyond the 
scope of this paper, but we reviewed the main publicly available data sources related to the 
pipeline to determine the degree to which these sources provide the information needed to 
improve candidates’ learning opportunities and outcomes. Information can support 
improvement efforts in two important ways, by a) allowing stakeholders assess to the level of 
performance on a valued outcome—or put another way, an overall level of success; and b) 
allowing stakeholders to identify where and for whom breakdowns in the system are 
happening. Assessing the level of performance on a valued outcome enables people to identify 
the gap between current and desired performance and can motivate improvement efforts. Data 
that helps identify where process breakdowns are happening and for whom helps system 
leaders to know where to focus their improvement efforts, and how to apply the limited time 
and resources they have towards high-leverage problems (Bryk et al., 2015). 


Using only the publicly available data sources about the California learning to teach 
pipeline we find that it would be difficult to answer some of the foundational questions a policy 
maker, researcher, or pipeline institution leader might find central to system improvement 
efforts. In particular, around these foundational questions, a stakeholder would have difficulty 
monitoring whether the system is producing the capable novice teachers that students need or 
whether parts of the system are functioning as intended. 


We begin by briefly summarizing the public data that are currently available. We then 
ask four foundational questions pertaining to the alignment of the system and its ability to 
prepare capable beginners. Subsequently, we describe how existing data and new data might 
be used to answer these questions. There are, of course, many other questions policy makers 
might need or want to ask of the system. The four questions we describe are illustrative 
examples that point to how the state might leverage data to provide the information necessary 
to support data-based decision making to improve the teacher education learning system. 
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Current Data 


The California teacher pipeline system produces a plethora of data, much of which is 
publicly available. The data include a variety of online PDF documents indicating accreditation 
decisions for TPPs and induction programs; online displays of various data on the supply and 
demand of teachers; dashboards that display the information that is required for federal Title II 
reporting; and results from the CTC’s annual statewide “Credential Program Completer 
Surveys.” These data, dashboards and reports provide pipeline institutions and policy makers 
with important descriptive data aggregated at a relatively high level, such as the number of 
certificates of a specific type granted by each CTC-accredited TPP and participation and average 
scores of program enrollees for the various teaching assessments, including a breakdown of 
scores by teacher demographics. A fuller description of the publicly available data can be found 
in Appendix E. Most of these data are available on the CTC’s website. 


The Data to Support Continuous Improvement 


Previously we argued that one criterion against which we might judge the teacher 
eduction system is the degree to which it provides critical information that is linked to the 
outcomes the system values. Two other critera—system alignment and the capabilty of 
beginning teachers — specify areas around which information for improvement would be very 
useful to stakeholders. 


We begin with a few notes. Consistent with the current data system and standards, any 
additional data should take extreme care to protect individuals and maintain their 
confidentiality. Pipeline institutions generally have experience with treating data confidentially, 
protecting its transfer, and using it for appropriate purposes. All additional data that 
institutions might share with or link to other data sets should be subject to the same scrutiny 
and careful treatment as existing data. In addition, given the current achievement levels of 
California’s students, we believe that any expansion of data collection or improvement in data 
reporting should focus on system improvement, as opposed to accountability. 


Given these caveats, we now turn to a consideration of whether the teacher education 
system is performing against the criteria we have introduced in this paper. We begin a focus on 
assessing system alignment, framed in terms of the first two illustrative questions. 


To what degree does ground level variation in institutions create unequal subject matter 
learning opportunities for prospective teachers in similar pathways? To help answer this question, 
we might start by looking at the different pass rates on various credentialing exams in teacher 
preparation. While we have noted that the CTC’s dashboard shows limited variation in the 
candidate pass rates on various knowledge assessments, a recent CTC report shows significant 
differences between first time and cumulative passing rates at the state level (CTC, 2016a). As 
noted in this same report, there are differences in passing rates for candidates of different 
racial and ethnic backgrounds. To understand why these pass rates differ or some students 
have to take the assessments multiple times in order to pass, it is necessary to look at the 
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nature of the key educational experiences within and across pathways to ascertain what might 
explain the differences. Are weaknesses in content knowledge a result of gaps in novices’ 
opportunities to learn the necessary content in their undergraduate institutions? Or did the gap 
begin back in their K-12 school? Or did the TPP not provide adequate learning opportunities? 


The specific details about what content a specific knowledge assessment measures and 
where in the pipeline that knowledge should have been developed, are important. For example, 
recall from Figure 2 that the CBEST is often required before entering a TPP. Thus, if candidates 
are struggling to pass CBEST, it makes sense to examine data from their higher education and K- 
12 experiences to pinpoint where learning opportunities were insufficient. Similarly, the CSET 
assesses a candidate’s subject-specific knowledge in the relevant certification area and such 
knowledge is frequently taught in K-12 schools and college majors. Conversely, RICA measures 
what a candidate knows about reading development. That knowledge likely comes almost 
exclusively from TPPs. Understanding where in the pipeline specific KSCs were not developed as 
intended and the specific groups of prospective teachers most affected, could help to target 
improvement efforts to the institutions most likely to be able to affect improvements. That 
ideally would prompt the next stages of inquiry to understand where high leverage changes 
could be made within the appropriate institutions to improve candidate content knowledge. 


Current reports that provide the scores on the first and cumulative testing attempts, are 
helpful for improvement. However, the data underlying these reports is not available publicly 
and it is unclear whether and how such data are used by pipeline institutions. Systematic 
institutional data on testing could be used formatively, to support institutions as they try to 
understand how remediation efforts such as courses and test preparation activities are (or are 
not) supporting novice learning. Information about the number of attempts candidates make 
before passing as well as information about those who never passed would also be useful. If 
such data could be linked to course taking patterns, college majors, or prior/future pipeline 
institutions, each institution could understand its role in supporting candidate learning and 
ultimate success. In addition, the state’s dashboards do not show key disaggregations — data at 
a level of detail that can help pipeline institutions understand the sources of the variation 
underlying the Title II data. For example, data about the extent to which types of 
undergraduate institutions, TPPs, or pathways help explain variation in number of testing 
attempts could be useful for targeting supports to the right groups of candidates and 
institutions. Additionally, pooling data across years makes it impossible to see how patterns 
change year over year. 


Linked, disaggregated data would allow better diagnosis of the sources of undesirable 
variation. For example, if there is consensus that fail rates on CSET exams are unacceptably 
high, linked disaggregated data could help reveal whether fail rates are related to candidates’ 
college majors, their course-taking patterns, or their high school experiences. Do candidates 
master KSCs early on and keep those KSCs differentially or did they never learn them at all? Do 
these learning opportunities vary by pathway (e.g., intern, traditional, etc.)? Policy makers, 
researchers, and leaders of pipeline institutions could all use this information to determine for 
whom and how to best improve content knowledge learning opportunities. 
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To what degree are novices working on the teaching skills most likely to improve student 
cognitive and non-cognitive development during induction? It is important for novices to be 
learning the KSCs most likely to support K-12 students’ develooment—both cognitive and non- 
cognitive. These skills are identified in the CSTPs. As a candidate leaves her preservice program, 
the TPP is responsible for preparing an ILP identifying areas of strength and weakness that 
serve as the starting point in that candidate’s induction program. Those plans are individualized 
to each candidate. As induction begins, the induction mentor and novice set measurable goals 
and work together to improve these specific areas of the novice’s teaching practice. Individual 
learning plans are maintained as individual level documents, and are not publicly available in a 
deidentified form. Nor, to our knowledge, are they aggregated to illuminate patterns of 
strengths/weakness across beginning teachers by TPP or employing districts. Knowing the 
variation in completers’ KSCs across specific practices could help TPPs and program sponsors to 
know whether there are certain areas in which their completers are consistently struggling; 
induction providers to know what supports and training mentors might need; and districts to 
know what types of professional development and supports might be particularly useful to 
novices. Finally, given the reality that many teacher labor markets are local, if there are 
differences between what TPPs and K-12 schools see as candidates’ weakness — e.g., schools 
notice that novices need stronger skills working with English learners but TPPs see candidates’ 
skills as adequate in that area—those differences are important to know and address. 


There are similar foundational questions stakeholders might like to answer with existing 
data that concern the degree to which California’s teacher education system is supporting the 
development of capable teachers. We now turn to the second two of four illustrative questions. 


Where do we lose our most capable and desirable teachers across the pipeline? We have 
documented that close to 40 percent of multiple- and single-subject credentialed teachers in 
the state are teaching with “substandard” credentials (intern credentials, waivers, and permits) 
and we know that many districts in the state experience challenges in staffing classrooms 
periodically, and the state’s most academically vulnerable students are disproportionately 
impacted by these shortages (Carver-Thomas & Darling-Hammond, 2017). The state collects 
and reports on trends related to supply and demand in the annual CTC report, which includes 
the number of teachers who received credentials, certificates, permits and waivers (CTC, 
2017d). Specifically, it includes information about how many new credentials were issued by 
type, pathway, and program sponsor, TPP enrollment data, demographic data, and estimated 
teacher hires by region, county, and subject areas, among other information. The CTC has also 
recently released an Educator Supply and Demand dashboard, which displays this information 
(in addition to other information) online (CTC, n.d.c., n.d.d.). While useful, these resources are 
missing the data to help diagnose the drivers of these shortages, which could then inform 
improvement efforts. While teacher shortages may be a demand-side problem, i.e. driven by 
retention rates that are lower than desirable, the state does not have linked data that allows 
ongoing monitoring of the dynamics and trends associated with teacher retention. For example, 
what types of districts struggle most with teacher retention? Are there any types of programs 
that systematically produce teachers who stay in the profession? If so, which ones and how can 
other programs learn from these successful programs? 
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Data that provide insight into the nature of the supply side of the teacher labor market 
are also unavailable publicly. Where are the leaks in the teacher education pipeline? If we take 
a cohort of young people who express interest in teaching on their undergraduate applications 
to CSU, who opts into Liberal Studies or other majors that are common in teacher pathways? 
Where and when do they drop out of those pathways? How many enroll in a TPP? For those 
that dropout of the pipeline, what can we learn about their experiences that might help plug 
the major leaks in the pipeline, especially for those who could be on track to take jobs in 
shortage fields or are candidates of color or bilingual candidates? 


Much of the data needed to answer these questions exist in administrative data sets 
held by TPP program sponsors and state data sets at CDE and CTC. The fields needed to link 
those data sets to one another also exist. However, the data sharing agreements are yet not 
established to link the necessary data. More than a decade ago the state planned for a teacher 
data system that would help with many of these questions, however, due to a series of 
technical and other challenges, the system was not built. 22 In recent years there has been an 
increased effort to at least link the data, even if there is no funding for a new online system.23 
Again, progress on this foundational question requires existing data to be linked across pipeline 
institutions and disaggregated to the levels deemed most useful for informing improvement. 


A second question about the capabilities of beginning teachers underscores the need for 
linked and disaggregated data, but also illuminates the need for a reorganization of existing 
data. 


Are there certain pathways that show evidence of producing more capable teachers? 
California offers different formal pathways into teaching (e.g. traditional, university intern, 
district intern) and is adding more, for example the Integrated Teacher Education Programsz24 
and a potential new residency in special education.25 A given TPP may play a role in several of 
these preparation pathways. Whatever their role, TPPs and K-12 schools provide essential 
opportunities to learn the specialized KSCs teachers need for teaching: integrated teaching 
practices, knowledge of English language development, and knowledge of reading 
development. Each program works on these integrated teaching practices with their own 
curriculum and programs verify candidate mastery over some of these practices through one of 
the three TPAs. 


What is not systematically available is information that might provide insight into the 
relative capabilities of the teachers trained through different pathways—though some of this 
information could be available with updates to current data collection and reporting. Take for 
example the new CTC’s Credential Program Completer Surveys, which collect statewide data 


22 https://edsource.org/2016/renewed-call-to-create-statewide-database-on- 
teachers/95339?utm_source=March+1+daily+digest+-+Michael&utm_campaign=Daily+email&utm_medium=email 
23 http://www.lao.ca.gov/Reports/2016/3355/prop-98-analysis-O21816. pdf 

24 https://edsource.org/2017/undergraduate-education-major-banned-for-56-years-returns/585830 

2s https://edsource.org/2018/gov-browns-california-budget-proposes-fully-paying-for-k-12-funding-formula-in-his- 
last-year/592432 
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from (1) individuals recommended for a teaching credential, (2) master teachers, and (3) 
employers. The teacher surveys are broken down by credential type (e.g., single-subject, 
multiple-subject) and include data on credentialed teachers’ perceptions of how effective their 
TPP was at developing the skills and tools necessary to become an effective teacher, with a 
specific focus on how well their TPP prepared them to adequately perform the six domains of 
teaching practice within the CSTPs. The master teacher surveys provide data on the 
perceptions of experienced teachers serving as mentors related to the extent to which the 
student teachers with whom they worked will step in to a classroom on the first day of work 
able to adequately perform the six domains of teaching practice within the CSTPs (CTC, 2017f). 
The employer surveys ask employers who have hired and seen at least two new teachers in- 
person over the past 3-5 years to complete a survey for a TPP from which those candidates 
graduated. The employer survey is focused on TPPs, not teacher candidates and asks about the 
degree to which program completers are prepared to teach in the six CSTP domains (CTC, 
2017g). Response rates for all these surveys are reportedly very high (CTC, 2017e).26 


Currently, data from the teacher, master teacher, and employer surveys are not 
disaggregated by type of preparation pathway (traditional, university intern, district intern) or 
program sponsor. Adding the option for disaggregation by pathway (and if needed some 
revision to the data collection to elicit responses that could be disaggregated by pathway), 
might provide some insight into the perceived effectiveness of these pathways and the 
strengths and weakness of candidates in specific pathways. 


Direct assessments of KSCs provide a different lens on capabilities. The CTC dashboards 
show pass rates broken down by pathway for CSET, CBEST and RICA. These dashboards even 
can be disaggregated by all those who have "completed non-clinical courses" and "other 
enrollees” -- in addition to completers. As noted however, what can be learned from the data is 
constrained by limited variation in the publicly reported data. Another significant gap in this 
information concerns what candidates can actually do—i.e. their mastery of integrated 
teaching practices. 


At the broadest level, there is evidence beginning teachers can carry out some practices 
that align to the CSTPs. This evidence comes from accreditation reports for TPPs and induction 
programs and the successful completion of a TPA. But there are many practices that can align 
with any one CS7P or TPE. There is no data that would help programs or other stakeholders 
know that all candidates can carry out any specific teaching practice before being allowed to 
serve as a teacher of record. Practices might include carrying out a parent teacher conference 
or modeling how to write a grade appropriate paragraph in ELA or managing a classroom of 
students through an inquiry-oriented science laboratory. Further, because professional choice 
regarding the specific practices candidates must demonstrate occurs in TPPs as well as 
induction programs, we know little about what teachers can do when they are granted their 


26 The employer survey response rate is reported as 100 percent but it is unclear whether this represents all 
programs in the state or simply that every employer filled out a survey for at least one program. 
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clear credential. While TPA data exist and are used to support recommending a novice for 
certification, it is not available publicly in ways that could help stakeholders improve a TPP, a 
cooperating K-12 district, or other pipeline institutions. To the extent that the data on the 
integrated teaching practices that candidates demonstrate before receiving a preliminary 
credential can be analyzed by pathway, there would be a particularly fruitful source of data 
because integrated teaching practices are meant to be developed during the TPP time. Pathway 
data about completers’ capabilities, measured by stakeholder surveys or direct assessment, 
could allow policy makers to be more strategic about when and how to expand, add, or 
discontinue certain pathways. 


In summary, our review suggests that there are important data than can be built upon, 
linked, specified at the level of teaching practices, and disaggregated to answer at least four 
foundational questions that could help California improve its teacher education pipeline. 
Gathering, manipulating, aggregating, or disaggregating data in ways that lead to wiser, more 
efficacious policy would support the state’s efforts to close the wide gaps in students’ learning 
opportunities and outcomes. While beyond the scope of this paper, we must also note that to 
ensure that new and improved data is used to drive improvement, the importance of timeliness 
and accessibility of the data cannot be understated nor can the importance of a safe 
professional environment for stakeholders to be transparent about where processes are not 
working well and take risks by making informed changes to key processes. 


Conclusion & Policy Implications 


This description and analysis of the teacher education system in California, which has 
focused on the learning progression through which candidates move from high schools through 
Community colleges, IHEs, California’s workplaces, preparation, and induction programs, 
arguably reflects the CTC’s conceptualization of teaching as complex interactional work carried 
out by well-trained professionals. 


Our analysis of these well-articulated learning pathway(s) suggests there are many 
features that can be built upon to maximize the state’s positive momentum in student 
achievement while also addressing the relatively low levels of student proficiency. In 
considering options for strengthening California’s teacher education pipeline, we return to the 
three criteria we used to judge the effectiveness of the system: the degree to which the 
pipeline has aligned policy messages for stakeholders, prepares capable beginning teachers, 
and generates relevant data. As noted, the fourth criterion, the supply of teachers is taken up in 
another Getting Down to Facts II white paper and the fifth criterion is beyond the scope of this 


paper. 
Aligned Policies 


We find that California’s teacher education pipeline is, at a high level, an aligned system 
of standards, assessments, and requirements oriented around a professional view of teaching 
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quality that is consistent with foundational research on teaching and learning. This alignment is 
the result of years of work that span more than 20 years. Policy makers should continue to 
protect and strengthen that alignment moving forward to maximize clear policy messages to 
stakeholders across the pipeline. 


It is important to note, however, that while the system is aligned, the state allowed over 
one-third of beginning teachers in 2016-17 to be the teacher of record before having met those 
aligned requirements (CTC, n.d.d). So the system may or may not function as intended if half of 
new teachers have not actually gone through the intended system. This is a critical area for the 
state to investigate systematically so that is can understand the positive and negative 
consequences of the increased use of substandard credentials on their ultimate audience, K-12 
students. It may be the case that such policies provide a much needed source of excellent 
teachers of color who stay in the profession or it may be that they generally serve a large group 
of teachers who do not complete the credential process and therefore are learning on the job, 
while students suffer for their inexperience and lack of skills. The state should investigate these 
and other potential outcomes with all due haste. 


Capable Teachers 


Our analysis suggests there is room for improvement in the area of the demonstrated 
capabilities teachers have on their first day teaching. There are five types of KSCs that beginners 
need for their first day, four of which focus on knowledge. The fifth KSC focuses on integrated 
teaching practices. We find there is standardization around the specific knowledge candidates 
are responsible for prior to entering the classroom, however there is much less standardization 
around the specific integrated teaching practices candidates are required to demonstrate 
before entering a classroom. This leaves stakeholders with high levels of assurance that 
teachers know certain things, but lower levels of assurance that all beginners can carry out the 
set of common practices all beginners need (Ball & Forzani, 2009). Contemporary research on 
teaching and learning has documented that knowledge is necessary but not sufficient for 
competent beginning teaching (c.f., Ball et al., 2008). It has also documented that practice- 
based aspects of teacher education are related to retention as well as where one student 
teaches (e.g., Ingersoll, Merrill, & May, 2014; Ronfeldt, 2012). California’s policies have already 
begun to work on this issue by mandating a TPA for every novice. But there is enough variation 
and choice in the specific practices and topics in which candidates demonstrate skill, that policy 
makers might profitably consider how to better balance knowledge and capability within the 
learning system. 


Teaching candidates primarily learn and practice integrated teaching practices in their 
TPPs and the districts in which they are placed for clinical work and subsequently in the districts 
in which they take jobs. Our survey evidence suggests that one area of improvement surrounds 
supporting general education teachers’ capabilities in working with students with special needs. 
This is being addressed in the new TPEs, however, it is likely too early to see statewide 
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improvement. Said in a more general way, another implication of our work is that policy makers 
could consider incentivizing and supporting TPPs, districts, and induction programs to clearly 
articulate the specific integrated teaching practices they want candidates to be able to carry 
out and then collect data indicating whether candidates have learned those practices. Given the 
limited number of practices any one TPA can assess, in order for programs to see marked 
improvement across the 47 elements of the TPEs — a necessary condition for large scale 
improvement, there will need to be additional standardization efforts within programs around 
the focus and measurement of these novices’ learning. Several CSU TPPs, in partnerships with 
their district partners, are already making strides in this area. 


Finally, to the extent the state considers options for strengthening support for the 
development of KSCs, we offer the reminder that the primary institutions where teachers 
would be expected to learn many of the KSCs are not always the TPPs, and so the roles of 
institutions along the entire the pipeline require attention. This is certainly true of California’s 
universities and community colleges who teach teachers their more advanced subject matter. 
And it is especially true for California’s K-12 schools, who 1) teach teachers the content they 
eventually teach children, 2) supply the learning context and mentors for student teaching 
experiences or intern experiences, and 3) provide the learning opportunities and mentors who 
guide beginning teachers to their clear credentials during the induction period. 


Relevant Data 


Our review suggests that the data infrastructure in the state is not optimally targeted 
towards supporting understanding of key valued outcomes. Further, we do not yet have a 
system of high quality, linked data that can be disaggregated to the levels needed to inform 
understanding and action. As a result, our teacher data is limited in the extent to which it can 
shed light on whether and how significant ground-level variation may be influencing the 
system’s overall success. It is also limited in its ability to support data-based improvement 
efforts. 


Given the breadth of institutions in the teacher education system and the urgent need 
for improved teaching across the state, we suggest developing systems and data with a primary 
focus on their ability to inform improvement efforts. There is ever-increasing empirical 
evidence in varied fields of social science that suggests high-stakes accountability policies carry 
unacceptably high risks of failure and problematic unintended consequences. To paraphrase 
the oft-cited Campbell’s law: The more a quantitative indicator is used for social decision- 
making, the more likely it will be to distort and corrupt the social processes it is intended to 
monitor. Thus, policy makers will need to make judgments about what information is most 
likely to provide useful improvement information for specific pipeline institutions, the CTC, 
citizens, and the state. 


A first step toward developing data sources that can support improvement might be 
clarifying specific valued outcomes, given the criteria for judging the system’s performance 
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introduced in this paper. Valued outcomes may shift over time, however, consensus around an 
initial set of valued outcomes could serve to focus improvement efforts in the near term. An 
organizational learning perspective suggests that the valued outcome itself is not enough to 
help focus improvement efforts and inform specific action that will lead to improvements--what 
is needed is insight into particular processes that make up the related systems. 


Learning from other states that have made strides in their efforts towards developing 
robust teacher pipeline data systems offers lessons California can build on. These lessons are 
about starting with the development of clear theories regarding how the state will use data to 
help achieve valued outcomes, engaging a range of stakeholders in the development of the 
systems, and taking an iterative, continuous learning approach to the development process 
(TNTP, 2017). California’s teacher learning system has many strengths that can and should be 
built upon in order to close the pernicious gaps the state’s children experience. _ If California 
chooses to work on developing a sense of valued outcomes and orienting the teacher learning 
system around producing information about the state of the teacher learning across the 
pipeline, the state will be well on its way to developing beginners who are ready to teach all 
children on the first day. 
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Appendix A 


CKTE Physics Test Question 8 of 21 


Three questions refer to the following information. Question 1 of 3 is on this screen. Questions 2 and 3 are on the 


following screen. 
Two students in Ms. Engel’s physics class are discussing the fala 


energetics of dribbling a basketball on a wooden floor. They agree Is Marcos correct that the elastic energy of the floor and the 
that all of the kinetic energy gets converted into elastic energy for ball would be the same? 

an instant when the basketball is compressed the most. They also 
recognize that many objects can be modeled as springs, even 


basketballs and wooden floors. They are uncertain about whether © ie. 

there would be equal amounts of elastic energy in the ball and the © No. The elastic energy of the ball would be greater. 
floor. They call Ms. Engel over to share their ideas with her and get f 

some help. © No. The elastic energy of the floor would be greater. 


Marcos says, “We were thinking that when the ball compresses © There is not enough information to compare these energies. 


against the floor, the forces that the ball and the floor exert on each 
other would be equal and opposite, so maybe the amount of elastic 
energy in the floor is the same as the elastic energy in the ball.” 


Louisa responds, “I get that the forces are the same, but | am 
thinking that the ball compresses more than the floor, so shouldn't 
there be more energy stored in the ball?" 


Marcos replies, “But the floor is more rigid and would have a higher 
spring constant. | think the larger k of the floor compensates for the 
smaller Ax in the %k(Ax)* equation, and the elastic energies are the 
same.” 
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CKTE Physics Test uestion 12 of 21 


In a situation with a number of interacting objects, one may select any subset of them as the system of interest. The objects that have 
not been selected as belonging to the chosen system are therefore external to the system. 


A bowling ball is dropped from a height of 1 m above a small trampoline. It lands and stretches the trampoline downward. Describe the 
energy changes in this process from the time the ball is dropped until it reaches the lowest point of the motion. 


For each of the following student responses, select the system for which the student response is correct. 


Record your answers by clicking on the appropriate cells. 


First there is gravitational 
potential energy. Then Earth 
does work and the ball gains 
kinetic energy. The 
trampoline does negative 
work on it and we have 
elastic energy. 


The gravitational potential 
energy is converted gradually 
to kinetic energy as the ball 
falls and ends up as elastic 
energy. 


First there is gravitational 


potential energy, which is 
converted to kinetic energy 
as the ball falls. Finally, the 
trampoline does negative 
work and there is no more 
kinetic energy. 


There is no mechanical 
energy at the beginning and 
then Earth does positive work 
on the ball and it gains kinetic 
energy. Then the trampoline 
does negative work on it and 
kinetic energy decreases to 
zero. 


There is no mechanical 
energy at the beginning, then 
Earth does positive work on 
the ball and it gains kinetic 
energy. Finally, all kinetic 
energy goes to elastic 
energy. 


Copyright © ETS, 2014 
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Appendix C 


CA High School Graduation Requirements 


California Education Code 51225.3 states that all pupils receiving a diploma of graduation from 
a California high school must have completed all of the following courses, while in grades nine 
to twelve, inclusive: 


e Unless otherwise specified, each course shall have a duration of one school year: 


(0) 


[e) 
[o) 
[e) 


Three courses in English 

Two courses in mathematics, including one year of Algebra | (EC Section 51224.5) 
Two courses in science, including biological and physical sciences 

Three courses in social studies, including United States history and geography; 
world history, culture, and geography; a one-semester course in American 
government and civics, and a one-semester course in economics 

One course in visual or performing arts, foreign language, or commencing with 
the 2012-13 school year, career technical education. For the purpose of 
satisfying the minimum course requirement, a course in American Sign Language 
shall be deemed a course in foreign language 

Two courses in physical education, unless the pupil has been exempted pursuant 
to the provisions of EC Section 51241 


e Other coursework adopted by the local governing board of the LEA 
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Appendix D 


RAND American Educator Panels 


ra American Educator Panels p 


Technical Documentation for the Stanford University’s Getting Down to Facts II study 
October 2017 Survey: RAND American Teacher Panel and American School Leader Panel 


Background 


Stanford University (Stanford) contracted with the RAND Corporation to field a survey to 
the RAND American Teacher Panel (ATP) in October 2017. The RAND ATP and ASLP is are 
nationally representative panels of K-12 public school teachers and principals who have agreed 
to participate in online surveys several times each school year. The panels include teachers and 
principals in every state to provide national estimates, as well as oversamples in twenty-two 
states to provide state-level estimates in these states. More information about the ATP is 
available at https://www.rand.org/education/projects/atp-aslp.html 


California is one of the states where teachers and principals were oversampled. The ATP 
California oversample was collected using a single-stage sampling technique. That is, teachers 
were sampled from a comprehensive database that served as a sampling from of all eligible 
teachers in CA. Note, however, that there was some moderate oversampling of inexperienced 
teachers and teachers of core subjects such as math and ELA — otherwise, teachers were 
sampled with equal probability. The ASLP California oversample was also collected in a single- 
stage manner; however, there was no oversampling of specific types of administrators. 


The surveys target teachers and principals in California. As of October 2017, the ATP and 
ASLP consisted of 800 teachers and 1037 principals, respectively. The RAND ATP and ASLP team 
invited all CA teachers and principals to take the respective survey. 
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Survey schedule 


Date/s Activity 


October 27, 2017 Field start date 


October 26, November 2-8- Reminder email dates 
14-21-28, December 13, 


January 3 
November 20, 2017 Reminder letters mailed to ASLP non-responders 
January 5, 2018 Field period closed 


Final Response Rates 


Of the 800 teachers sampled, 459 were determined complete and weighted in the final 
data set. The final response rate for teachers was 57.4 percent. Of the 1037 teachers sampled, 
318 were determined complete and weighted in the final data set. The final response rate for 
principals was 30.7 percent. For the ATP survey, anyone who responded to more than 10 
percent of the items for which they were eligible received a weight. This low threshold was 
used due to the relative lack prevalence of item nonresponse on this survey. For the ASLP 
survey, the threshold was 20 percent. 


Sampling Weights 


Sampling weights were generated for the Stanford survey and these weights are 
provided in the data file. There are two primary reasons to weight the RAND ATP and ASLP 
data. 


First, the weights help to adjust for nonresponse (at both the panel recruitment and 
survey stages). So if, for example, we had relatively fewer elementary school teachers respond 
than middle school teachers, the weights will reconcile this response differential. 


Second, Stanford’s interest is in a representative sample of California teachers and 
principals. The ATP and ASLP have oversampled certain types of educators (e.g., inexperienced 
educators) for a variety of reasons throughout the development of the panels. Weights also 
adjust for such oversampling. 


Survey weights are calculated by first determining initial weights, which (briefly 
speaking) are based on sampling and enrollment/response probabilities. These initial weights 
are then calibrated so that the weighted sets of survey respondents match the national 
population of teachers and principals based on several school-level (e.g., school size, level, 
urbanicity, socio-demographics) and individual-level (e.g., gender, education, experience) 
characteristics. Weights are calibrated to match known national-level totals for these 
characteristics that are found using the National Center for Education Statistic’s Common Core 
of Data and Schools and Staffing Survey reports. 
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To facilitate estimation of uncertainty levels (e.g., standard errors) through the use of a 
jackknife, replication weights were calculated. The ATP and ASLP were each segmented into 80 
jackknife replication groups. Each group has 1/80* of the respective panel excluded, and each 
panelist is excluded from one (and only one) group. A separate set of weights is calculated for 
each replication group by applying the weighted processes above to the respective group (while 
ignoring any panelist excluded for the respective group). Therefore, there are 80 sets of 


replication weights. Note that panelists excluded from a replication group receive a weight of 
zero for the respective set of replication weights. 
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Appendix E 


Publicly Available Data Sources 


TPP Accreditation Reports 


First, state-level data include publicly available annual reports from the CTC Committee 
on Accreditation. These reports list TPP accreditation decisions for all programs that received a 
visit or revisit from the CTC Committee on Accreditation. However, in order to understand why 
a TPP did not receive full accreditation but, rather, accreditation with stipulations, major 
stipulations, probationary stipulations, or with a 7“ year report, one must visit a separate CTC 
accreditation report website. Moreover, TPPs that gain CTC accreditation do not necessarily 
meet all Common and Program standards required by the CTC Committee on Accreditation. To 
understand which Common and Program Standards a TPP has not fully met, one must visit the 
separate CTC accreditation report website and click on individual TPP accreditation reports. 
Accessing this data provides insight into whether a TPP is lacking in preparing teacher 
candidates in some KSCs that teachers need on their first day in a classroom. For example, in a 
review of one TPP that was accredited with stipulations, we learned that the School of 
Education did not have consistent, accurate, timely or reliable advisement of teacher 
candidates, an important aspect of a teachers’ preparation. Accreditation reports do not 
contain candidate outcome data that could be used to judge the degree to which a program 
met its goals. Such data might include, for example, candidate perception data or information 
about the progress through the program of teachers of color or the demonstrated 
improvement of learning opportunities to address gaps in candidate’s KSCs. 


The accreditation reports include select disaggregated program level data that is useful 
for some purposes but not others. Depending on the state’s goals, the current reports may or 
may not be useful. It is worth noting that at the state level, there is no single database that 
would allow a researcher, policy maker, the public and other interested stakeholders to easily 
explore, compare and contrast TPPs’ success in meeting the CTC Committee on Accreditation’s 
preconditions, Common Standards and Program Standards.27 Such information would be 
necessary for stakeholders interested in streamlining or strengthening accreditation 
requirements or even understanding how programs respond to various accreditation 
recommendations. 


27 Preconditions are requirements grounded in statute, regulations and/or Commission policy. Common Standards 
address aspects of program quality that cross all approved educator preparation programs within an institution 
and demonstrate that the program sponsor has sufficient infrastructure to support each program’s successful 
implementation. Program Standards address the quality of program features that are specific to preparation for a 
given credential, such as curriculum, field experiences, and knowledge and skills to be demonstrated by candidates 
in the specific credential area. 
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Induction Accreditation Reports 


We were not able to locate annual (or occasional) state-level CTC induction 
accreditation report similar to the TPP accreditation high-level reports that aggregate 
information above the level of program; however, the accreditation reports of individual 
induction programs are available.zs Just as with the TPP accreditation reports, the induction 
reports specify whether the program met accreditation standards and what the specific issues 
were, if standards were not met. The reports also contain detailed program information — e.g., 
what curriculum is used, whether the program partners with the New Teacher Center, the 
nature of the K-12 student community, etc. The reports also note how many teachers the 
program has served. The accreditation reports we reviewed did not contain any systematic 
information that linked these details to valued outcomes, e.g., beginning teachers’ views of the 
support they receive in the program or documentation of beginners’ learning over time. Just as 
with the TPP accreditation reports, the current reports might be useful, depending on the 
state’s goals. As they stand, they might be useful for learning what is in a specific program as 
well as who the program serves. In their current state of disaggregation, however, they would 
not be useful for understanding induction program strengths and weaknesses at a system-level 
or even within a particular geographic area. If any large scale analysis were to be done, all of 
the text information would need to be put into a database capable of serving that purpose. 


Annual Teacher Supply Report 


A third source of publicly available information is the annual CTC report on the number 
of teachers who received credentials, certificates, permits and waivers (CTC, 2017d). The 
report’s goal, which is mandated by AB 471, is to provide essential information on the teacher 
supply. Specifically, the report provides details “on the number of teachers who received 
credentials, authorizations, permits and waivers. [...] The report responds to the requirements 
specified in statute and provides a tool for policymakers and others interested in teacher 
supply.” (CTC, 20174, p.1). It includes information about how many new credentials were issues 
by type, pathway, and program sponsor, TPP enrollment data, demographic data, and 
estimated teacher hires by region, county, and subject areas, among other information. The 
CTC has also recently released an Educator Supply and Demand dashboard which displays this 
information (plus some additional information) online (CTC, n.d.c., n.d.d.). 


Title Il Dashboard and Reports 


A fourth source of publicly available teacher pipeline data is the state Title II data 
dashboard, which displays data that are mandated pursuant to the Title II of the Higher 
Education Opportunity Act. The online dashboard provides topline data as it relates to GPA 
requirements; distributions of TPP completers by preparation pathway (traditional, university 
intern, district intern) and subject area; pass rates for required assessments; gender and 
ethnicity distributions of students enrolled in TPPs, by preparation pathway; and supervised 


28 See https://info.ctc.ca.gov/fmp/accreditation/accreditation reports.php#LEA for induction reports at the 
bottom of the site, under “district intern programs and other programs”. 
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clinical experience details (e.g., average number of clock hours of supervised clinical experience 
required at different time points), by preparation pathway. Data from the annual CTC report on 
teacher credentials are also provided in the Title Il data dashboard file. There is an annual 
report to the legislature required by Title II (Public Law 110-315, 2008) that aggregates these 
data to the state level. There is also downloadable data for the last two years that can be 
disaggregated by all of the variables listed for the topline data. 


One piece of data that was not available in the previously described CTC report on 
teacher credentials but that is available in the Title Il report is the number of credentialed 
teachers broken down by academic major. This data is a good step towards elongating the 
teacher preparation pipeline reporting to include data on teacher candidates’ undergraduate 
education. Yet, the data is disaggregated only at the preparation pathway level (traditional, 
university intern, district intern), not, for example at the TPP level. As such, it is not possible to 
explore questions such as the extent to which a teachers’ high school experience or 
undergraduate major, for example, are related to her success in completing her preparation 
program or induction requirements. 


Credential Completer Surveys 


A fifth source of publicly available data already mentioned is the CTC’s “Credential 
Program Completer Surveys,” which are data from statewide surveys of (1) individuals 
recommended for a teaching credential (2) master teachers and (3) employers. The teacher 
surveys are broken down by credential type (e.g., single subject, multiple subject) and include 
data on credentialed teachers’ perceptions of how effective their TPP was at developing the 
skills and tools necessary to become an effective teacher, with a specific focus on how well 
their TPP prepared them to adequately perform the six domains of teaching practice within the 
CSTPs. Response rates for the teacher surveys are high (CTC, 2017e). The master teacher 
surveys provide data on perceptions of the extent to which the student teachers with whom 
they worked will step in to a classroom on the first day of work able to adequately perform the 
six domains of teaching practice within the CSTPs (CTC, 2017f). The employer surveys ask 
employers to complete a survey for a program from which they have hired at least two teachers 
in the past five years. The employer survey is focused on preparation programs, not teacher 
candidates and asks about the degree to which program completers are prepared to teach in 
the six CSTP domains (CTC, 2017g). The employer survey response rate is 100 percent but it is 
unclear whether this represents all programs in the state or simply that every employer filled 
out a survey for at least one program, which could mean there is a good deal of missing data for 
small programs or programs for whom the administrator did not hire at least two teachers in 
the past five years. 


Data from the teacher, master teacher, and school leader surveys are not disaggregated 
by type of preparation pathway (traditional, university intern, district intern) or program 
sponsor. Moreover, the surveys specifically ask how well TPPs prepare teachers, and do not 
address the extent to which a teacher’s undergraduate or high school education may also 
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contribute toward their preparation as a teacher who can adequately perform some domains of 
teaching practice within the CS7Ps. 
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